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Since 1993, the America Youth Policy Forum 
(AYPF) has studied and highlighted strategies and 
reforms that provide youth with high quality 
education and preparation for fulfilling careers. Part 
of AYPF’s mission is to publicize best practices in 
the education and youth development fields to help 
policymakers and practitioners make informed 
decisions. As a non-partisan professional 
development organization, AYPF explores many 
options supported by a variety of philosophical 
underpinnings. Central to our approach is our focus 
on whether positive outcomes are achieved by 
young people. Our focus has not been explicitly on 
“high performing schools,” “high performing 
programs” or “high performing administrators and 
staff’ but on high performing young people. We 
consider schools, programs, administrators and staff 
to be high performing when they have positive 
effects on young people. 

Understanding this focus of AYPF’s work on high 
performing youth provides an important guide to 
using this volume. AYPF has spent five years 
collecting empirical evidence of youth outcomes and 
compiling them into readable volumes. This report 
continues our commitment to placing sound 
research and evaluation at the service of 
policymakers and practitioners as they wrestle with 
some of America’s most enduring challenges — 
achieving true equality of educational opportunity 
and equity in educational outcomes. 

In 1997, AYPF published its first compendium of 
summaries of evaluations of programs and practices 
that were found to be successful in propelling youth 
to rewarding careers and postsecondary education, 
reducing risky or illegal behaviors, and providing 
opportunities to youth who had dropped out of 
school or were leaving the juvenile justice system. 
That report, called Some Things DO Make a 
Difference for Youth: A Compendium of Successful 
Youth Practice and Programs , was so well received 
that AYPF produced a second volume, More Things 
That DO Make a Difference for Youth in 1999. In 
this era of increased national attention to academic 



achievement, many of the profiled programs in these 
two volumes were able to document academic 
achievement gains, as well as other positive outcomes. 

Funding from the William T. Grant Foundation 
allowed for a re-analysis to determine exactly what 
the evaluations in the previous compendia could tell 
us about outcomes related to academic 
performance. This analysis is published in Raising 
Academic Achievement: A Study of 20 Successful 
Programs (2000) — programs with both the strongest 
achievement gains and the strongest evaluations. 
Five of the 20 programs directly addressed the 
question of minority student success. In particular, 
Alliance for Achievement (no longer in operation) and 
Gateway to Higher Education (currently expanding) 
illustrated (1) the long way minority students still have 
to go to eliminate the academic achievement gap, 
because despite their intellect and initiative, they are 
underepresented in higher level courses, SAT test- 
taking, college enrollment, and other avenues to higher 
achievement; and (2) how much these programs 
helped increase the numbers of minority students at 
higher and higher levels of achievement. 

Again with William T. Grant Foundation support, a 
much longer journey was begun to find evaluations 
of educational programs that are working to raise 
the academic achievement of minority students. On 
this journey there were a few surprises, including 
that many well-known programs we had hoped to 
document had no evaluations. Often, we found 
evidence of success from other, less well-known 
programs. We learned of the width of the academic 
achievement gap between African American, Latino 
and Native American youth and their white and Asian 
peers, yet were encouraged by the many programs that 
recognized the gap and were working hard to increase 
academic achievement for their young people. 

Since beginning this volume, we have witnessed an 
escalating concern about the “minority achievement 
gap.” Some have even called it the education issue 
of the new millennium in policy circles and the 
media. We hope that this volume can help provide 
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guidance on what works for minority youth to reach 
higher levels of academic achievement. 

A Few Words about our Focus on Academic 
Achievement 

This report focuses only on academic achievement , 
not on the broader range of indicators of the first 
two compendia, such as employment and earnings 
data and reductions in risky behavior. In our 
compendia, the focus is on “hard data” primarily to 
“prove” the effectiveness of these programs, 
especially to those who are skeptical of softer 
measures. This decision coupled with the 
presentation of brief summaries of each evaluation, 
means that much rich information about other 
outcomes for youth may have been omitted. 

In the first two compendia, information was 
provided on a range of strategies used by 
successful programs. The report, Raising 
Academic Achievement, narrowed the focus to what 
the program evaluations had to say about one set of 
outcomes. The current volume narrows the focus 
further to academic outcomes for minority 
students. Yet, within this academic achievement 
category, the focus is deep, seeking outcomes along 
an optimal pathway of academic achievement we 
wish all young people could take. 

It is our hope that all young people will — 

♦ attend school , arrive on time , go to all classes 

♦ read at grade level or above 

♦ do well in the sciences, mathematics and 
technology 

♦ persist to high school graduation 

♦ be appropriately identified and served for any 
special needs 

♦ obtain good grades (C or higher) 

♦ have access to and do well in academically 
challenging courses 

♦ have opportunities to apply their knowledge 
while in school (through work-based learning 
or service-learning ) 

♦ follow a coherent course sequence leading to 
postsecondary education 



♦ take standardized and college entrance exams 
(e.g. Stanford 9, California Achievement Test, 
SAT, ACT, Achievement, and Advanced 
Placement tests) and obtain competitive scores 

♦ make thoughtful guided decisions about 
college attendance and financing 

♦ enroll in college 

♦ have no need for remedial education in college 

♦ sustain academic achievement and good 
grades in college 

♦ sustain financial aid (reapply as needed) 

♦ sustain college enrollment 

♦ graduate from college 

♦ and successfully pursue graduate/professional 
school degrees or fulfilling work in their 
chosen career 

Information was sought for each level of 
educational achievement along this pathway. 
Information was also sought in the youth 
development literature about youth that are not in 
school, but there was little data on academic 
achievement to be found here. There was however, 
a great deal of writing on the topics of minority 
over-representation in special education, 
misidentification for special education, under- 
representation in gifted and talented programs, 
Advanced Placement, Honors and other advanced 
classes, and over-representation in the juvenile 
justice and adult penal systems. The research being 
conducted on these areas of concern is of great 
importance to the issue of minority academic 
achievement. However, summarizing the research 
on these topics was far beyond the scope of this 
report. 

Additionally, within the pool of evaluations that met 
our rigorous criteria, there were few that provided 
detailed descriptions of the programs evaluated and 
the strategies used. So, while it is possible to 
identify strategies believed to be effective, 
practitioners wishing to implement these strategies 
or seeking to influence other types of youth 
achievement should use the contact information 
provided in each of the program summaries. 
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Background 

Raising Minority Academic Achievement: A 
Compendium of Educational Programs and 
Practices reports on a 22-month effort to identify, 
summarize and analyze evaluations of school and 
youth programs that show gains for minority youth 
across a broad range of academic achievement 
indicators, from early childhood through advanced 
postsecondary study. 

Purpose 

The purpose of this report is to inform policymaking 
and funding decisions by providing easy-to-read, 
accessible, concrete, research-proven evidence of 
academic achievement gains for minority youth and 
information on successful program strategies. The 
report also aims to provide information that 



researchers, practitioners (school administrators, 
youth program directors, teachers, counselors, 
youth workers), families, community members and 
young people can use to evaluate, design, 
implement and advocate practices shown to be 
effective in raising minority academic achievement. 

Programs 

An exhaustive search of journals, research 
databases, and other sources yielded over 200 
documents pertaining to education programs. To be 
included in the report, these documents had to use 
sound methodology and have measurable academic 
achievement data on racial or ethnic minorities. 
Those documents that met the criteria for inclusion 
were summarized in three to five pages and 
subjected to a review process that resulted in 38 
being chosen for final inclusion: 



Abecedarian Program -Chapel Hill, NC 
Advancement Via Individual Determination (AVID) - 
nationwide 

Alaska Onward to Excellence & Alaska Rural Systemic 
Initiative- AK 

Boys & Girls Clubs of America - CA, FL, NY, OH, TX 

Calvert - Baltimore, MD 

Career Academies - nationwide 

Chapel Hill-Carrboro City Schools - NC 

Chicago Arts Partnership in Education -Chicago, IL 

Chicanos in Higher Education - nationwide 

Child-Parent Centers -Chicago, IL 

City Schools - nationwide 

Class Size: Project SAGE - Wl 

Class Size: Project STAR - TN 

Class Size Reduction - CA 

Compact for Faculty Diversity - nationwide 

Dare to Dream - FL, IN, MN, TX 

Emerging Scholars Program - nationwide 

Equity 2000-CA,MD, RI,TN,TX 

Gateway to Higher Education - New York, NY 



\ 

GE Fund College Bound - in 1 2 states 
Head Start & African American Children - nationwide 
Head Start & Latino Children - nationwide 
High School Puente - C A 
High Schools That Work - in 23 states 
High/Scope Perry Preschool- Ypsilanti, Ml 
Historically Black Colleges and Universities - in 20 states 
I Have a Dream - Chicago, IL 
KIPP Academies -Houston, TX& Bronx, New York 
Project GRAD - CA, GA, NJ, OH, TN 
Puerto Rico Louis Stokes Alliance for Minority 
Participation - PR 

Sacramento START - Sacramento, CA 
Sponsor-A-Scholar- Philadelphia, PA 
Success for All - nationwide 
Texas District-Wide Initiatives -TX 
Tribal Colleges - in 1 0 states 
Upward Bound - nationwide 

Urban Elementary Schools -G A, IL, MA, MD, Ml, TX, Wl 
Vouchers - DC, NY, OH 

) 
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Outcomes 

♦ Early Childhood — Evaluation findings were 
particularly strong and positive at the early 
childhood level. When compared to control 
groups, minority children who attend early 
childhood development programs are more 
likely to remain in school, complete more years 
of education, and require less special education. 

♦ Elementary Through Middle School — The 
elementary through middle school evaluations 
were almost exclusively focused on test scores. 
In most cases, improvements were incremental 
and even where minority academic achievement 
increased, the disparities in achievement 
between minority and white youth were highly 
apparent. Texas is probably the only state 
where achievement gaps between minorities and 
white students are being halved or cut even 
more. However, Texas students are measured 
on passing rates on only a minimum 
competency test. The question of whether 
higher levels of achievement are eventually 
reached remains unanswered. 

♦ District or State Initiatives/K-12 — The report 
reviews several district or state initiatives, 
including class size reduction and voucher 
studies. Additionally, reforms in Texas, North 
Carolina and Alaska are reviewed. These 
evaluations tend to focus more on increased 
attention to accountability than on specific 
strategies used to increase minority academic 
achievement. 

♦ High School/Transition — Because they focus 
on more than test scores, the high school/ 
transition programs offer a better perspective of 
what is actually happening with their minority 
students. Among the positive findings from 
these programs were one or more of the 
following: increased high school graduation, 
more high school credits earned, higher GPAs 
earned or maintained, more college prep and 
Advanced Placement courses taken, increased 
enrollment in higher level mathematics and 



science classes, more college entrance exam- 
taking and higher scores, less need for 
remediation in college, higher levels of college 
enrollment at two- and four-year colleges, 
higher levels of college retention and 
graduation, and continuation in science- 
related majors or professions. Success is a 
relative word for most programs. Students 
may be entering college at a higher rate but 
their GPAs may be similar to peers in regular 
classrooms, or more students may be enrolling 
in academically challenging courses but also 
failing these courses in higher numbers. 
Evaluations of Upward Bound, Sponsor-A- 
Scholar and Career Academies show that 
improvements were most significant for those 
with higher risk of school failure and/or lower 
initial expectations, especially as they stayed in 
the program longer and participated more 
intensely. However, selective programs, such 
as Gateway for Higher Education and High 
School Puente , also indicate that high achieving 
students can perform at still higher levels when 
challenged. 

♦ Postsecondary — Fewer quality evaluations 
were available at the postsecondary level with 
data disaggregated by race or ethnicity. The 
postsecondary programs included in the report 
show African American, Latino and Native 
American youth succeeding in demanding 
careers and postsecondary education. 

However, their numbers are still quite small. 

Strategies 

The school initiatives and youth programs included 
in this report provide concrete examples of efforts 
to increase achievement for minority youth. 
However, no “magic bullet” was found, that is, no 
one strategy was found to guarantee program 
success. Rather, it is recommended that 
practitioners implement a comprehensive set of the 
following strategies and continuously evaluate their 
effects. The ten most frequent strategies identified 
in this report are listed below from most to least 
frequently cited in the program evaluations. 
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♦ Program quality . Quality of implementation, 
leadership and accountability are three essential 
components of effective strategies that help 
ensure high program standards. 

♦ Academically demanding curriculum . All 
early childhood programs included in this report 
provide preschool-aged children with 
challenging educational activities that are also 
developmentally appropriate. Concern with 
challenging curricula was equally apparent in K- 
12 programs. 

♦ Professional development Many of the 
evaluations report professional development 
activities including staff orientation, summer 
sessions, ongoing training during the school 
year and/or when changes in curriculum or 
school structure are implemented. Programs 
that rely on tutors or mentors offer them 
training and supervision. 

♦ Family involvement Approximately 40% of 
the evaluations report activities geared toward 
improving communication with families, or 
increasing family involvement with the 
programs. Although such efforts are concentrated 
on initiatives for young children, at least two high 
school programs also include activities to promote 
greater involvement of families. 

♦ Reduced student-to-teacher ratios . Many 
programs showing academic gains for minority 
students include a range of strategies to reduce 
student-to-teacher ratios, including smaller 
classes, small learning communities, teacher 
aides, team teaching, tutoring, mentoring and 
other ancillary supports. 

♦ Individualized supports . For students who are 
struggling academically, individualized support 
may be the difference between falling behind 
and moving ahead. Many programs utilize 
community members, college students, 
employers and other groups as tutors and 
mentors to address the academic needs of 
specific students, or offer support, feedback 
and encouragement. 



♦ Extended learning time . Several programs use 
longer school hours, extra school days, 

Saturday and summer courses to provide 
students with more learning time. 

♦ Community involvement Several programs 
involve communities, both individuals living 
close to the program and the larger community 
such as employers, museums and artists. 
Community participation takes many forms, from 
reinforcing cultural traditions and knowledge, to 
advocating for improved academic achievement of 
minority students, to offering work-based learning 
opportunities for students. 

♦ Long-term (multiple-year) supports for youth. 
Several programs encourage long-term, stable 
relationships between participants and 
knowledgeable adults, from two to five years in 
most cases. 

♦ Scholarships and/or financial support 
Several K- 12 programs offer financial help to 
students who demonstrate high academic 
performance. 

Recommendations 

Based on AYPF’s reflections on the reported 
evaluations, following are actions policymakers, 
practitioners, researchers, parents and community 
members can take to improve minority academic 
achievement. 

1 . Focus on Improved Academic Achievement and 
Outcomes for All. 

♦ National leaders should continue to build 
consensus around acceptable achievement 
gains and require that these gains be shown for 
all student groups. National attention should 
focus on achievement differences among the 
states and ways to eliminate these differences. 

♦ States should create benchmarks for 
improving academic achievement for all 
student groups and provide resources for 
school districts to attain those benchmarks. 
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♦ States and school districts should support and 
maintain high quality leadership and ensure 
the adequate implementation of programs to 
enhance minority academic achievement. 

♦ School districts and schools should expect 
high achievement from all students and 
provide academically demanding curricula that 
are meaningful and available across schools and 
grade levels to bring all students to higher levels 
of knowledge and achievement. 

♦ States and localities should develop a multi- 
layered “check” of achievement using a 
variety of test measures, such as NAEP, state- 
mandated tests, Stanford-9 or ITBS; and also 
use indicators that provide a broad perspective 
about students, such as classroom-based 
assessments, attendance, behavior (disciplinary 
incidents), course enrollment and passing rates, 
types of courses completed and graduation 
rates, among other measures. 

♦ School districts and schools should provide 
professional development and support to 
ensure that teachers (and other involved adults, 
as appropriate) have a deep understanding of 
curriculum, are familiar with innovative 
instructional methods, and have knowledge and 
interpersonal competence with cultures other 
than their own. 

♦ Schools should provide students , families and 
communities with specific information on 
what constitutes high academic standards and 
support their expectations for excellence in the 
educational system. 



2. States and Localities Should Provide the 
Necessary Supports to Ensure Student Success, 
including: 

♦ Reduced student-to-teacher ratios . A range of 
strategies should be employed by schools and 
programs to provide more personal teaching and 
learning environments to foster higher levels of 
academic achievement. These strategies may 
include smaller classes, small learning 
communities, teacher’s aides, team teaching, 
tutoring, mentoring and ancillary supports. 

♦ Extended learning time . To accelerate and 
reinforce student learning, programs should 
encourage or require additional time and 
opportunities (such as longer days, weekends 
and summer courses). 

♦ Long-term supports . Programs should 
encourage student participation over an 
extended time (two years or more) to create and 
sustain stable relationships between participants 
and knowledgeable adults, and to help youth 
make successful transitions as they progress up 
the educational ladder. 

♦ Scholarships and/or financial support . 
Programs should provide financial support to 
youth as needed to motivate participation and 
persistence in quality educational experiences. 
Programs should also provide continual 
guidance to youth and monitor the impact of the 
funds on student achievement, retention and 
graduation. 

3. Start Early, Don’t Stop. 



Families , youth advocates and communities 
should hold schools accountable for high 
levels of achievement for all students, reinforce 
academic skills learned both at home and at 
school, and ensure that every child has an 
advocate outside of the school system or 
program. 



National leaders , states and school districts 
should prevent minority students from falling 
behind by expanding early childhood programs 
and providing continuous guidance and supports 
through the elementary and high school years. 

National leaders , states and school districts 
should boost efforts to increase minority 
students’ entry into and graduation from 
postsecondary education . 
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A Note on Educational! Research most useful to policymakers and practitioners 

and (c) provide guidance to practitioners on 
The introduction to Section II, in which the how to initiate and use program evaluation. 

Evaluation Summaries are presented, describes the 

obstacles and discoveries along the way to selecting ♦ Public and private funding sources should 
the 38 evaluations included in this report. require and support high quality program 

Observations garnered from the work of creating evaluations and utilize findings to improve 

this report include the following: policy and practice. 



♦ Finding useful evaluations of educational 
programs is a difficult task, particularly when 
criteria for assessing quality are used. 

♦ The most useful research is based on simple but 
methodologically sound design and provides 
information that is clear and easy to understand. 

♦ Without rigorous research, program 
practitioners may be perpetuating failing or 
mediocre interventions whose long-term 
consequences are costly to young people and 
society. 

♦ Disaggregating data for analysis is essential to 
highlight areas that require improvement, as 
well as areas of proven success. Programs that 
claim overall success without disaggregating 
their data may be helping one group of students 
while masking the low achievement of other 
groups. 

♦ Evaluations frequently “spin” results into 
“success” or hide less than successful results, 
rather than present a thoughtful and balanced 
analysis of what worked and what did not. 

Based on the experience with this and the previous 
compendia, recommendations for improving 
educational research in the area of program 
evaluation include: 

♦ A large-scale, national and comprehensive 
educational research agenda should be 
developed to (a) determine which strategies and 
policies have resulted in the most benefit, for 
whom, and at what cost, (b) provide guidance 
to evaluators on what type of research would be 



♦ Data should be disaggregated by race, 
ethnicity, limited English proficiency, 
disability status, gender and poverty level and 
be made publicly accessible to researchers, 
educators, policymakers, families and the public 
at large. 

♦ Researchers should look into a range of 
achievement indicators including, numbers of 
students enrolled and dropping out, attendance, 
test scores, GPAs, graduation, suspensions, 
expulsions, and special education referrals. 

They should also translate their findings into 
language that is accessible to policymakers, 
practitioners, educators, families and students, 
so that research findings can be translated into 
more effective education policies and practices. 

Conclusion 

At almost every educational level, schools and 
community-based programs across the country are 
reporting good news about the academic 
achievement of the minority students they are 
serving. Although gaps overall are still large, and 
most reported achievement gains are small, these 
programs have proven there is every possibility of 
succeeding in raising achievement for all. 
Implementing the recommendations above could 
help the nation move beyond a feeling of 
helplessness regarding achievement gaps by 
providing specific information on program design 
and strategies about “what works” to enhance 
academic achievement. The larger challenge is 
creating the national will to set in place 
mechanisms that will eliminate differences in 
academic achievement among students correlated 
with race or ethnicity. 
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Raising Minority Academic Achievement: A 
Compendium of Education Programs and Practices 
is the culmination of a 22-month effort to identify, 
summarize and analyze evaluations of school and 
youth programs that show gains for minority youth 
across a broad range of academic achievement 
indicators from early childhood through advanced 
postsecondary study. 

The purpose of this report is to inform policymaking 
and funding decisions by providing easy-to-read, 
accessible, concrete, and research-proven evidence 
of academic achievement gains for minority youth, 
and information on successful program strategies. 
The report also aims to provide information that 
researchers, practitioners (school administrators, 
youth program directors, teachers, counselors, 
youth workers), families, community members and 
young people can use to evaluate, design, 
implement and advocate practices effective in 
raising minority academic achievement. 

This report is divided into two major sections. 
Section I contains four chapters. Chapter 1 

BSeseaiirdh) Gtoft© 

Following is a detailed description of the process by 
which AYPF chose the 38 educational initiatives 
that appear in this report. 

i. Acceptance Criteria 

At the outset of this project, a search was set in 
motion to collect evaluations of programs and 
initiatives aimed at improving the academic 
achievement of minority youth. Before 
initiating the search, the editorial team 
established the following criteria to guide the 
acceptance of documents: 

♦ Population - The evaluations had to 

contain data on racial or ethnic minorities as 
defined in the adjacent box. 



provides background and summary data on minority 
academic achievement and, as the title suggests, 
raises the question, is there — Achievement for 
All? Chapter 2, Measuring Academic 
Achievement , introduces the 38 education 
initiatives summarized in Section II, and describes 
the measures and levels of academic achievement 
for minority youth reported by evaluators. 

Chapter 3, The Search for the “Magic Bullet , ” 
describes the most prevalent strategies used by 
programs in which minority youth made 
significant academic achievement gains. Chapter 
4, Moving Forward , provides recommendations 
based on the report’s findings. 

Section II contains the 38 three- to five-page 
summaries of program evaluations and studies in 
alphabetical order. The summaries are preceded by 
an introduction, A Journey Through Educational 
Research , which reflects on the difficulty of finding 
evaluations meeting the criteria for inclusion and 
makes several observations regarding educational 
research. The Glossary defines research terms used 
in the report. 

f ' \ 

In this report, the term “minority” is used to 
identify racial/ethnic groups in the United 
States other than whites of European origin. 

The report uses the U.S. Census terminology 
for “minorities” including African Americans, 
Asians/Pacific Islanders, and Native 
Americans/ Eskimos, but adopts the broader 
category “Latinos” rather than “Hispanics.” In 
addition, Asians/Pacific Islanders has been 
shortened to “Asians” and Native American/ 
Eskimos to “Native Americans” except in the 
summary of the Alaska Rural Systemic 
Reform program. The terms African American 
and Latino are used in this report even when 
evaluators used the terms Black or Hispanic. 
Although an effort was made to cover all 
groups, more information was found on African 
American and Latino youth than on other 
minority populations. 

\ J 
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♦ Measurements - The studies had to include 
measurable (quantitative) data related to 
academic achievement of minorities. 
Preferably, they would present a set of 
measures including: school attendance; 
grades; credits completed; test scores on 
state mandated tests and/or national 
achievement tests (such as SAT, ACT, 
Achievement Tests and Advanced 
Placement Tests); high school graduation, 
college access, retention, and receipt of 
undergraduate and graduate degrees. 

♦ Methodology- Since expected findings were 
quantitative, the evaluations should adhere to 
accepted standards for quantitative research. 
Therefore, the following requisites were 
delineated: (a) research design - experimental 
or quasi-experimental, pre- and post- 
treatment, and longitudinal studies; (b) 
research period - the study should cover at 
least one school year; (c) researcher - 
preferably independent, that is, not directly 
associated with the program’s funding source 
or implementing organization to avoid bias; (d) 
sample - randomized sampling procedures, 
control and comparison groups should be 
matched to the treatment group by 
demographics and level of academic 
achievement; and (e) the data should be 
analyzed statistically with levels of significance 
not to exceed 5% (for discussion about the 
methodology used in the evaluations, see 
Section 77, Introduction). 

♦ Period - Preferably, programs and initiatives 
should be current. For this reason, the search 
was limited to evaluations conducted within 
the past five years, with two exceptions: 
ongoing longitudinal studies, such as the 
Abecedarian report, and studies that are still 
frequently cited in discussions of initiatives, 
such as the Tennessee STAR research. 

♦ Scope - In a departure from previous AYPF 
compendia dedicated solely to successful 
programs and practices, we decided to 
include studies of large, well-known 



programs and relevant federal initiatives that 
fit these criteria, even if they had mixed or 
negative findings. Another group of 
acceptable studies were qualitative research 
that provided a voice to minorities on the 
factors that they saw as influencing their 
academic success. 

2. Search Strategies 

The search for evaluations included the 

following sources: 

♦ Large databases, including the 
Educational Resources Information 
Center (ERIC) and library collections. 

♦ Internet search of over 50 associations 
and research centers dedicated to 
education and minorities. 

♦ Direct contact with program coordinators, 
policymakers, funding officers, and 
researchers. 

♦ Distribution of flyers requesting 
evaluations during forums, conferences 
and similar events. 

♦ A request for evaluations posted at the 
AYPF web site ( http://www.aypf.org ). 

The search also relied on the expertise of the 
project’s Advisory Board to indicate relevant reports 
and researchers who specialized in this field. 

3. The Review Process 

The written summaries passed through a review 

process divided into four steps: 

♦ Internal review - The editorial team 
reviewed all summaries, making comments, 
and suggesting changes or documents to be 
eliminated. 

♦ External review - The summaries approved 
in this first review were then sent to an 
external reviewer to assess once more the 
quality of the research, propose 
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improvements, and suggest further 
elimination of weaker documents. At this 
stage, members of the Advisory Board also 
had the opportunity to read the summaries 
and make comments and suggestions. 

♦ Researchers * review - After another round 
of editing, the summaries were provided to 
all the program evaluators and directors to 
review for accuracy. 



♦ Final review - The AYPF directors and the 
editorial team read the summaries once 
more for final editing and approval. 

Of the more than 200 documents reviewed, 38 
made the final cut. Although this report is a 
collaborative effort, it should be reiterated that the 
final decision on which summaries to include and 
the opinions expressed in the report are the sole 
responsibility of AYPF. 
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Schooling is a top concern of many Americans, 
including the subject of presidential and legislative 
debates. No matter how wide ranging the issue, the 
overriding question is: “How can we raise academic 
achievement?” 

This question has been approached with increasing 
gusto since the 1983 A Nation at Risk 1 report 
decried “mediocrity” in education and has 
intensified after results from the Third International 
Mathematics and Science Study (TIMSS) showed 
U.S. students trailing students from other developed 
nations. The reaction has been a heightened interest 
in testing student achievement and a flurry of 
education reforms, many of which have not been 
subjected to strict analysis and rigorous evaluation. 

We know from a number of indicators that progress 
is being made in advancing academic achievement 
in American schools. In 2000, Do You Know the 
Good News about American Education? reported 
positive information about our public schools, 
including decreases in high school dropout rates; 
increases in the number of students taking more 
challenging courses; improvements in mathematics 
and science achievement; increased SAT and ACT 
test scores; more students taking Advanced 
Placement classes; more students going on to higher 
education; and more Americans completing four- 
year college degrees. 2 

However, there is evidence that these gains are not 
evenly distributed across populations of students. 
Are higher average indicators hiding pockets of low 
performance or large gaps in achievement? While 
this question is relevant to many categories of 
students (e.g. across gender, socio-economic status 
and disability status) the focus of this publication is 
on racial and ethnic minorities. 

Aire we keeping the promise? 

In effect, A Nation at Risk set the bar of high 
achievement — “Our goal must be to develop the 
talents of all to their fullest.” It also provided the 



caveat: “that a public commitment to excellence 
and educational reform must [not] be made at the 
expense of a strong public commitment to the 
equitable treatment of our diverse population.” 
Finally, it honed in on the repercussions of failure to 
include all young people in these elevated 
expectations. The concern with excellence was 
maintained in the educational legislation that 
followed, including the Goals 2000: Educate 
American Act , the Improving America's Schools 
Act and others. States have also enacted legislation 
requiring high standards for all students and more 
accountability for public schools. 

The question is, are we keeping the promise? — a 
promise echoed over and again in challenges to 
“leave no child behind” and reflected in the 
collective voice of many education leaders that 
minority academic achievement may be the most 
important educational and social issue of the 
century. 3 

Despite the encouraging statistics on educational 
achievement for young people in the aggregate, 
there is no denying that, for the majority of African 
American, Latino and Native American youth in the 
United States, the educational system is not fulfilling 
its promise. In fact, when data is disaggregated by 
race or ethnicity, disparities appear. Assessments of 
kindergarteners already show that African American 
and Latino children are over-represented in the 
lowest quartiles of achievement tests. 4 

As minority children move through their school 
years, the differences persist. For the past 30 years, 
minorities (except for Asians) have scored 
consistently lower than whites on all National 
Assessment of Educational Progress (NAEP) tests. 5 
For instance, the average 1975 NAEP reading 
scores for 9-year-old African American and Latino 
students were about 30 points lower than the 
average scores for white students. After some 
improvement in the early 1980s, the gap in 1996 
increased again, as shown in Figure 1. For 17-year- 
old students, the 1980s represented a period of 
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Fig. 1 - Average NAEP reading scores 

of 9-year-old students 
by race/ethnicity: 1975-1996 




[|§j Hispanic 
| | African American 

Source: Adapted from U.S. Department of Education , 
National Center for Education Statistics , The Condition of 
Education 2001, NCES 2001-072, Washington, DC: U.S. 
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improvement, with gaps in average scores being 
reduced by 40 points for African Americans and 
more than 20 points for Latinos, but again falling, 
although less sharply, in the 1990s (Figure 2). 
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Fig. 2 - Average NAEP reading scores 
of 17-year-old students by 
race/ethnicity: 1975-1996 



200 




1975 1980 1984 



1988 1990 1996 



■ White 
|H Hispanic 
|~1 African American 
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National Center for Education Statistics, The Condition of 
Education 2001, NCES 2001-072, Washington, DC: U.S. 
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score of white students. Asian students outscore 
white students by 35 points on the math test, but 
have a mean that is about 30 points lower on the 
verbal test. 7 



It is true though that more Americans are graduating 
from high school now than 30 years ago, and the 
graduation gap between white and minority students 
has narrowed significantly. In 1971, 82% of whites 
in their mid-twenties had graduated from high 
school compared to 59% of African Americans and 
48% of Latinos. In 1999, white and African 
American high school graduation rates were much 
closer at 93% and 89%, respectively. However, the 
Latino high school graduation rate still lags far 
behind both white and African American high 
school graduation rates at 62%. 6 

Similarly, SAT scores reveal an increase in minority 
academic achievement in recent years, yet a gap 
remains. The gap is largest for African American 
students, whose mean scores on the math and 
verbal sections of the SAT are approximately 100 
points lower than the mean score of white students. 
Latino and Native American students have less of a 
gap, between 45 and 75 points lower than the mean 



Although college access for minority students has 
increased in the past 30 years, an achievement gap 
still remains. Between 1971 and 1999, the 
percentage of white high school graduates who 
completed a bachelor’s degree or higher increased 
13%, from 23.1% to 36.1%. In this same period, 
the increase was only 5% for African Americans, 
from 11.5% to 16.9%, and 4% for Latinos, from 
10.5% to 14.4%. 8 As Figure 3 indicates, in 1999 
whites were twice as likely to obtain a bachelor’s 
degree than their African American and Latino 
peers. Asians out-performed all other subgroups in 
the completion of postsecondary degrees, except for 
the associate’s degree. 

In summary, minority youth have showed steady 
gains in many academic indicators in the past 
decades, but they still have a long way to go to 
reach parity with their white peers. Explanations 
about the reasons for this discrepancy are many and 
agreements are few. On average, minorities are 
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Fig. 3 - Percentage of population 25 years 
and older with postsecondary education by 
race-ethnicity (1 999) 
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Source: U.S. Census Bureau. Current Population 
Survey, March 1999. Educational Attainment in the U.S., 
table JO. Online at www.census.2ov/population/ 
socdemo/education/p. 20-528/tab JO. pdf 



starting from much lower baselines, at least in part a 
reflection of long years of segregation and 
discrimination. Inequalities in income, school 
resources, and the quality of teachers have also 
been frequently cited. A discussion on the many 
theories about the academic achievement gap is 
beyond the scope of this report. However, as this 
publication shows, when programs and policies 
emphasize academic achievement and provide 
quality supports, minority youth rise to the 
occasion. 



Why is minority academic achievement 
such an important issue? 

Although non-Latino whites constitute more than 
70% of the total U.S. population, 9 the term 
“minority” disguises the fact that the proportion of 
non-white students in America’s public schools is 
rising and already represents the majority of 
students in many localities. As Figure 4 indicates, 
between 1972 and 1998, the proportion of minority 
students in public schools increased from 22% to 
38%. For Latinos, the proportion more than 



doubled from 6% to 15%. Enrollment varies 
according to regions and in the West and South 
minorities already constitute 47% and 45% of the 
student population. The increase in the proportion 
of “minority” youth means that the prosperity of the 
nation will be increasingly dependent on the 
knowledge and contributions of minority young 
people. 

From speculations about our nation’s poor 
performance on international tests of knowledge to 
a real lack of skilled workers, it is increasingly 
apparent that every American counts. As the 
United States turns overseas to recruit more and 
more workers for highly skilled job openings, we 
abandon our own undereducated youth at our own 
peril. 



Moreover, failure to deliver on the educational 



promise only alienates young people from schools 
and other social institutions. For example, as Kati 
Haycock, Director of Education Trust comments, 
“Many young people are totally undone by the gaps 
between high school and college. They do 
everything their high schools tell them to do to get a 
diploma. But when they show up at even the local 



Fig. 4 - Percentage of public school 
students enrolled in grades K-12 who are 
minorities by region. October 1972-1999. 
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community college, they do not have the knowledge 
and skills necessary to begin credit-bearing courses.” 10 

Over-represented among the poor, minority youth 
are often in schools with the fewest resources, the 
least qualified teachers, and the least challenging 
curricula. Low-level achievement leads to less 
prestigious employment, lower wages, and poorer 
housing conditions served by the poorest schools. 
This cycle, well known to many minority families, 
further contributes to a feeling that school has little 
promise for change. 

New research is more hopeful that decreasing the 
educational gap between racial and ethnic minorities 
and white Americans will reduce economic gaps as 
well. In 1972, Christopher Jencks argued that 
reducing educational inequalities in America would 
not reduce economic inequalities. However, in 
1998, the findings of Jencks and Meredith Phillips 
suggest that due to the progress America has made 
on other social reforms, particularly in the 
workplace, the effect of increasing minority 
academic achievement on earnings and other 
measures of social equality would be more 
substantial than in 1972. On the topic of the 
academic achievement of African Americans, they 
found that “the test score gap between blacks and 
whites turned out to play a much larger role in 
explaining racial disparities in educational attainment 
and income than many had realized.” If “racial 
equality is America’s goal,” the authors write, 
“reducing the black-white test score gap would 
probably do more to promote this goal than any 
other strategy . . .” n Other minorities would similarly 
benefit from reductions in the achievement gap. 

Where have we documented the gap? 

The focus on standards and accountability in 
educational reform has led to efforts to disaggregate 
data and share the results widely. Without this 
attention to detail, the public would know less about 
the width of the minority academic achievement 
gap. According to a report from North Carolina, 

“the facts about the ‘educational condition’ of 
minority children have been known by education 
leaders for years. Despite having the facts, there 



has been a reluctance to tell parents, policymakers 
and the public the truth about how schools are doing 
in educating students of color.” 12 

Exposing the gap may force school districts to 
eliminate it. Daniel Domenech, Superintendent of 
Fairfax County Public Schools, VA, has stated that 
an advantage of Virginia’s Standards of Learning 
(SOL’s) has been to pinpoint disparities between 
schools within his district and to help him advocate 
for resources targeted to the areas of greatest 
educational need. 13 Also, the trend towards 
collecting, disaggregating and sharing data has given 
a new empowering tool to youth, parents and 
community members in demanding better school 
experiences and outcomes. For the first time, it is 
apparent exactly how much minority children are 
denied. 

While some states have collected enrollment data by 
race/ethnicity for years, most are just beginning to 
grapple with the extent to which educational 
inequities remain. Texas was the first to report 
achievement data publicly and require that schools 
show achievement gains not only for the student 
population as a whole but also for each subgroup. 

In addition, 2000 was the first year in which the 
federal Title I compensatory education programs, 
designed to address the special needs of children in 
high poverty schools, required all states to collect 
and publicly report disaggregated achievement data 
by race and ethnicity. 14 Individual Title I school 
administrators and teachers will be held accountable 
for ensuring that each racial/ethnic group as well as 
the school as a whole is making significant 
educational progress against some external standard 
(usually a standardized test based on state standards 
of learning). 

As school districts continue to disaggregate and 
make public their achievement data, a complex 
picture of educational differences is emerging, 
wealthy well -resourced suburban communities have 
been “shocked” to discover that even in their 
comfortable middle and upper-middle class 
communities, with a measure of economic equality 
and high achievement on average for their youth, 
goals of academic achievement for all have not 
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been met. For instance, a suburban New York 
school district, with a reputation for diversity and 
tolerance, has recently released statistics 
disaggregated by race. The data led parents, 
African American and white, to accuse the school 
district of systemic segregation including steering 



African American students away from honors 
courses and into special education, disciplining them 
at disproportionate rates, and allowing their test 
scores to lag far behind those of whites for a 
decade . 15 



Raising Academic Achievement vs. 
Reducing the “Achievement Gap” 

Much of the discussion on raising the academic 
achievement of minority students focuses on 
reducing the “achievement gap” between white 
and Asian students, on one hand, and African 
American, Latino and Native American students, 
on the other. For non-Asian minority students, a 
policy that focuses solely on closing the 
achievement gap has several pitfalls: 

First, gaps may close because the performance 
of higher achievers falls, and equity is achieved 
through the lowest common denominator. 

Second, gaps may stay the same because the 
performance of all groups increases. Or, gaps 
may also increase, because even though all 
groups perform better, the program has a 
stronger impact on high achieving groups. For 
example, the GE Fund College Bound program 
was successful at raising the college enrollment 
of all students, but white students experienced 
greater gains. Though this was a positive 
outcome, it actually increased the college 
enrollment gap between white and African 
American participants. 

Third, focusing on reducing the gap in one state 
may obscure pronounced academic achievement 
differences between states. For instance, using 
2000 NAEP data, reducing the 25 point gap 
between Latino and white students in California 
would increase Latino scores to 227, only 3 points 
above Latino scores in Texas, yet still 1 9 points 
below white scores in Texas (see Table). Therefore, 
policymakers should work to decrease minority 
academic achievement gaps, while also setting high 
academic achievement goals for all youth. 



Average scores of students in selected 
states at the 4 th grade 2000 NAEP 
mathematics assessment 



Race/Ethnicity 


TX 


CA 


White 


243 


227 


African American 


220 


191 


Latino 


224 


202 


Asian/Pacific Isl. 


246 


226 


Average 


233 


214 



Note: A ten point difference in the test corresponds 
roughly to one year of learning. 

Source: National Center for Education Statistics, 

The Nation ’. s Report Card. 2000 Mathematics 
Assessment, Grade 4 Public School Students. 

Percentage of Students and Average Mathematics 
Scale Score by Race/Ethnicity. Available at htto:// 
nces. ed.nov/nationsreDortcard/nae.ndata/ . 
/ 

Fourth, gaps may appear to close because the 
focus has only been on the students still in 
school, with no regard for the youth who drop out 
of the system. Some educators are concerned 
that the use of “high stakes” tests as a graduation 
requirement may encourage less prepared youth 
to drop out of school, thereby removing them 
from the test-taking population. 

Finally, the idea of raising academic 
achievement recognizes the need for changes, 
but says little about the overall quality of the 
education provided. The challenge is to define 
“quality” education and determine the 
benchmarks against which students’ performance 
will be evaluated. This discussion merits 
continued national attention but is beyond the 
scope of this report. 
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Raising Minority Academic Achievement 

Chapter 2; Hfeasyring Academic 

Achievement 



This chapter addresses the question of “what is 
happening in programs and initiatives that aim to 
improve the achievement of minority youth?” We 
discuss the findings of the 38 evaluations chosen for 
this report, taking a journey through the school 
experience of minority youth, from early childhood 
to postsecondary education. The report relies on 
measures and indicators imposed by states, school 
districts or researchers. Their findings are based on 
different populations and varying program 
objectives and strategies. Recognizing these 
limitations, no attempt is made to create a common 
denominator to define “success” or to compare 
programs among themselves. 

Early Childhood Programs 

Overview 

This report includes five summaries of four early 
childhood programs. The Abecedarian Project and 
High/Scope Perry Preschool were experimental 
preschools funded in the 1960s and 1970s to serve 
low-income, African American children. Both are 
no longer in operation, although the High/Scope 
curriculum is used in preschools around the country. 
Child Parent Centers ( CPC) is an ongoing Title I- 
funded program with multiple sites in high-poverty 
Chicago neighborhoods that are not served by Head 
Start. Head Start is a federal program established in 
1964 as part of the federal government’s “War on 
Poverty.” It provides matching funds to localities 
for comprehensive programs that offer low-income 
children, ages 3 to 5, with supports and stimuli to 
improve their chances of academic success. 

All summaries describe longitudinal studies of 
participants. The evaluations of the Abecedarian 
Project , High/Scope and CPC compare program 
participants to matched control groups, following 
the two groups through more than 20 years. The 
small sample sizes (except for CPC with a sample 



size of 1500), determined in part by the longitudinal 
nature of the studies, leave the conclusions open to 
questions. While it is difficult to identify precisely 
what factors influence an individual’s behavior over 
20 years, the duration of these evaluations offers a 
rare view of the potential impact of early 
interventions on participants’ lives. The two //ead 
Start evaluations review ten years of national 
databases. The 1995 evaluation compares the 
impact of the program for African American and 
white children, and the 1999 evaluation compares 
Latino and white children. 

Analysis 

Findings are quite similar in all five early childhood 
evaluations. When compared to control groups, 
children who attended childhood development 
programs are more likely to remain in school, 
complete more years of education, and are less 
likely to attend special education. Attending 
Abecedarian , for instance, cut in half the likelihood 
of participants receiving special education. Lower 
grade retention rates are cited in CPC and Head 
Start. The 1995 Head Start evaluation refers to a 
nearly 50% reduction in the likelihood that a 
program participant will repeat a grade in elementary 
school. Participation in Head Start was found to 
cut between one-quarter and one-third of the 
Latino-white score gap on the vocabulary, math and 
reading sections of the Peabody tests. 

This improved schooling may partially explain the 
positive social and work outcomes for program 
participants. High/Scope and CPC evaluations 
report a decline in arrests for youth who attended 
early childhood programs, while High/Scope and 
Abecedarian report that participants, now in their 
twenties, have more skilled, better paid jobs. ( CPC 
reports that men benefit more than women.) 
According to the CPC evaluation, longer attendance 
produces stronger results. 
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In contrast to long-term gains reported in the High/ 
Scope and Abecedarian studies, the 1995 Head 
Start evaluation found a decline in the academic 
gains of African American children after leaving the 
program. The benefits gained from Head Start 
were gradually lost and, by age ten, African 
American participants retained no gains, while white 
participants still retained an overall gain of five 
percentage points. The evaluators hypothesize that 
differences in the two sub-groups of children explain 
the loss of gains, since African American children in 
Head Start are more likely to be poor, live in poorer 
neighborhoods and attend schools with fewer 
resources than their white peers. 

Critics contend that evaluations of early childhood 
programs have biased samples, since parents who 
take time and effort to enroll their children in these 
programs are already more involved than parents of 
children outside the programs. This may be true, 
although it is a leap to imagine that all children who 
are not in early childhood programs have 
uninvolved or uninterested parents. Many reasons 
affect a parent’s decision not to use an early 
childhood program, from lack of programs near 
their neighborhoods to cultural tradition. As the 
1999 Head Start evaluation found, Puerto Rican 
children who remained home did better in school 
than those who went to Head Start or other 
preschools. Remaining home in this case was not 
an indication of inadequate parental motivation or 
involvement with the child. True randomized 
control-treatment groups bypass this discussion, but 
such groups are difficult to define in real life. It is 
also true that early childhood programs can only do 
so much for an individual’s life and that many other 
factors will contribute to one’s success or failure 20 
years later. 

Even with such caveats, the evaluations of early 
childhood programs show a strong pattern: such 
programs increase the chances for low-income 
children, including minority children, to do well in 
school and in life. In education, as in the health 
care field, investing in prevention is a cost-effective 
strategy. However, as no health care system can 
rely solely on preventive care, no education system 
can be satisfied without good K-12 schools to 



maintain and expand the educational gains of the 
early years. 

Elementary Through Middle School 
Programs 

Overview 

The majority of evaluations focused on the early 
elementary years, with only a few presenting data 
for grades six to eight. After-school programs were 
included in the search for evaluations, but for most 
of them, academic achievement was only a minor 
focus of a broader social mission, mainly to offer 
children a safe and supportive environment. Two 
after-school programs had evaluations with enough 
data on academic achievement and strong enough 
methodology to justify their inclusion in this report: 
Boys and Girls Clubs of America (B&GCA) and 
Sacramento START. 

B&GCA is a private, not-for-profit organization with 
clubs nationwide. The evaluation focused on an 
academic enrichment program offered to school- 
aged children who live in public housing projects. 

The program showed statistically significant 
increases on a variety of measures for program 
participants. During the 18-month evaluation, 
participants’ school attendance rates nearly doubled 
and their average grades increased from three to six 
points in different subject areas, while the 
comparison groups showed a decline in both 
measures in the same period. Sacramento START is 
an after-school enrichment program for elementary 
school children in low-income neighborhoods 
financed by the City of Sacramento, CA. The 
evaluation used school district data and matched 
comparison groups. It showed some improvements 
for all students, with striking improvements in test 
scores for students who had started the program 
with the lowest grades. 

Among the school programs included in this 
category, Calvert \ s a traditional, highly structured 
elementary school program transplanted from a 
predominantly white, middle-class private school 
into an all- African American low-income public 
school in Baltimore City. The evaluation uses a pre/ 
post-treatment design with three cohorts. Before its 
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implementation, no first grade in the public school 
had scored above the second quartile on the 
Maryland state tests. Three years later, the 
percentage of students scoring above the second 
quartile was 42%. For third graders, only 6% had 
scored in the third quartile before the program; one 
year later, 38% of the students had reached this 
quartile. The program’s 97% attendance rate was 
among the highest in the city. 

The Chicago Arts Partnership in Education 
( CAPE ) proposes an innovative approach to learning 
that involves arts in all subjects, taught by teams of 
teachers and artists. The program targets low- 
income K-12 schools with large numbers of African 
American and Latino students. The evaluators 
found a 50% increase in sixth grade scores on the 
Iowa Test of Basic Skills (ITBS) and nearly a two- 
year increase in the reading level of ninth graders, as 
measured by the Test of Achievement and 
Proficiency (TAP) for CAPE students between 
1992-1998. 

KIPP Academies are charter schools that serve low- 
income African American arid Latino students from 
grades five to nine. The academies offer a 
curriculum that focuses on “high standards” and 
college preparation. Within two years, the passing 
rates on the Texas Assessment of Academic Skills 
(TAAS) for KIPP students in Houston, TX, 
increased from 33% in mathematics and 63% in 
reading to approximately 100%. The KIPP 
Academy in the Bronx, New York, has been 
frequently rated the highest performing middle 
school in the area in terms of average attendance, 
reading and mathematics. 

Success for All is a reading program that has 
become one of the largest elementary education 
initiatives in the country. It uses small reading 
groups based on skill level rather than age, one-on- 
one tutoring, and a structured school day. Of the 
many evaluations of Success for All, this report 
includes a recent review of the TAAS database. 
Project GRAD is a comprehensive K-12 school- 
wide reform that uses a mix of strategies, including 
Success for All and others. The evaluation focuses 
on Texas schools and compares test scores for 



Project GRAD students with students in matched 
schools. Urban Elementary Schools reports on nine 
schools across the country that are increasing the 
scores or passing rates of minority students on 
different tests, including the TAAS. 

All three evaluations show increases in the 
percentage of students passing the TAAS. Success 
for All students show higher rates of improvement 
in comparison to students statewide, and a three- 
fourths reduction (from 25% to 6%) in the TAAS 
score gap between African American and white 
students from 1995 to 1998 (statewide, the gap was 
reduced from 25% to 14%). Project GRAD 
doubled the TAAS passing rates, particularly in 
math. In addition, it reduced disciplinary referrals 
by 74%. Urban Elementary Schools describes a 
school in San Antonio (Baskin Elementary) that 
eliminated the gap in passing rates for African 
Americans and Latinos within four years. Another 
school in Houston (Lora B. Peck Elementary) 
raised passing rates for Latino students on the 
writing section of TAAS from zero to 90% in the 
same period. 

Analysis 

Unlike the early childhood programs that follow 
students to the next level of schooling, the 
elementary through middle school evaluations 
appear more compartmentalized, providing 
information only within the elementary through 
middle school boundaries. The school adopting the 
Calvert Program is showing incremental 
improvements in the Maryland state test, although 
scores are still below the state’s satisfactory levels in 
all grades and subjects. 16 No research on Success 
for All was found that follows students beyond 
elementary school grades. Therefore, it cannot be 
determined whether improvement in these test 
scores is reflected in better performance at the high 
school level. 

In most cases, improvements appear quite modest 
while the disparities in achievement are striking. Texas 
is probably the only place where achievement gaps 
between minorities and white students are halved or 
cut even deeper, but these students are being measured 
on passing rates on a minimum competency test. 
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As the report indicates, many schools are working 
hard to improve achievement indicators for all 
students and not only a privileged few. The schools 
that are improving their students’ academic 
performance are starting from extremely low levels 
and through incremental gains are approaching a 
point closer to the middle. How these programs 
affect minority students who are already beyond the 
middle point is not clear. This observation is not a 
criticism of those schools or their districts and 
states. On the contrary, these schools deserve 
kudos and support for making a concerted effort to 
raise the achievement of their students. Moving the 
students from unsatisfactory levels to basic is a 
good start. However, the ultimate objective must be 
to bring all students, including minority students to 
much higher levels of knowledge. 

Distract or State Omfiatives (K-12) 

Overview 

Among the large initiatives covered in this report are 
three statewide projects on reduced class sizes 
( Project STAR in Tennessee, Project SAGE in 
Wisconsin and Class Size Reduction in California); 
the evaluation of three city wide experiments with 
vouchers (Voucher Schools)', a statewide initiative 
for Eskimo and Native American students in Alaska 
Onward to Excellence/Alaska Rural Systemic 
Initiative; a review of the statewide accountability 
reform in Texas, focusing on four school districts 
(Texas School Reform)', a compilation of data on 48 
urban public school systems nationwide (City 
Schools); and a district wide initiative to improve 
the academic achievement of African American 
students in North Carolina (Chapel Hill-Carrboro 
City Schools). 

Project STAR was a groundbreaking study on the 
impact of reduced class size on academic 
achievement, mandated by the Tennessee legislature 
in 1985. The evaluation involved 7,500 children in 
grades 1 to 3 and compared children taught in 
classes of 17 students per teacher with children in 
larger classes with and without a teacher’s aide. 
Evaluators found that students in small classes did 



better than both control groups on all tests. The 
effect size of small classes on African American 
students was double that of white students. A 
follow-up study of Project STAR followed 
participants from grades 4 through 6 and reported 
ongoing, albeit small gains (effect sizes of 0.2 or 
less) for students who were taught in small classes. 
(See Glossary for an explanation of “effect size.”) 

A decade later, Wisconsin implemented the SAGE 
project, a pilot study involving more than 3,000 
kindergartners and first graders statewide. In 
addition to using a control group in regular 
classrooms (30 students), the evaluation also 
compared different strategies to reduce student-to- 
teacher ratio, small classes being one of them. As 
with the Project STAR, evaluators found increases 
in test scores for all students, particularly African 
American students in the first year of the project. 

In the second year though, the score gap between 
African American and white students had increased 
again. Different from Project STAR, the SAGE 
evaluation found that score gains were not limited to 
small classes. Other strategies that reduced student- 
to-teacher ratio, such as team teaching or floating 
teachers, were equally effective. 

Unlike Wisconsin and Tennessee, California decided 
to forgo a pilot program; instead, launching a 
massive, statewide Class Size Reduction ( CSR) 
initiative that affected approximately 1 .8 million 
students by its third school year of implementation 
in 1998-99. The state funded the initiative on a per 
pupil basis only after small classes had been 
implemented. Therefore, in the first years of 
implementation, schools that did not have the 
facilities to create small classes — often high-poverty 
schools with large populations of minority 
students — received an average of $100 less per 
student than wealthier, predominantly white schools. 
When these schools did create new classrooms, 
they often did so at the expense of existing facilities 
used for special education, child-care, music, art, 
computers and libraries. After three years, the 
evaluators noted small (but statistically significant) 
achievement gains, with no differential impact for 
minorities. 
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School Vouchers analyzes three privately funded 
experiments to test the impact of vouchers on 
students in urban school districts with high 
percentages of minority youth (Dayton, OH, New 
York City and Washington, D.C.). The vouchers, 
awarded by a lottery system, covered only part of 
the private school tuition with the recipients’ 
families paying for the remaining tuition costs. 
African Americans constituted about 70% of the 
approximately 3000 students who received vouchers 
in the three experiments. Using the California 
Achievement Test (CAT) as the measure of student 
performance, the evaluators found a reduction for 
voucher recipients of approximately one-third of the 
test score gap between African American and white 
students. There was no positive or negative effect 
of statistical significance for any other ethnic group 
in the study. When controlling for family 
background, the overall difference between voucher 
and non-voucher students was not significant in 
Dayton and New York City, 17 but was significant at 
the .01 level in D.C. 

The Alaska Onward to Excellence/Alaska Rural 
Systemic Initiative began a decade ago through a 
partnership between public schools, universities, and 
Eskimo and Native American communities in rural 
Alaska. Most participant districts involve small 
fishing villages with difficult access. The project 
incorporated the cultural traditions of the native 
population with an academically demanding 
curriculum. One of the evaluations compares the 
scores of students in a single project district on a 
number of standardized tests (ITBS, CAT, ACT) to 
scores statewide, where white students are the 
majority. A ten-year trend analysis verified a steady 
increase in all standardized test scores for 
participating students. In the ACT test, for 
instance, the district experienced an increase in the 
number of seniors taking the test and a reduction of 
about 14% in the score gap between local seniors 
and the state average. The percentage of project 
students attending college rose from 10% in 1988- 
89 to 50% in 1996-97. 

Texas requires that a specific percentage of students 
in each school pass the state assessment in reading, 



writing and mathematics skills. Schools that do not 
attain this percentage risk losing their state 
accreditation. As part of the state reform, an 
emphasis has been given to monitoring the 
performance of minority students. The Texas 
School Reform summary covers four school districts 
with diverse populations. TAAS passing rates 
increased for all students in the four districts, but 
the increase for African Americans and Latinos was 
steeper. For instance, in the Aldine district, with 
83% minority students, between 1994 and 1999, 
passing rates for African Americans almost doubled 
(from 36% to 73%) and the rate for Latinos 
increased by 63% (from 49% to 80%). In the same 
period, white students’ passing rates increased by 
29%, from 68% to 88%. Similar findings are 
shown for the other districts. Evaluators did not 
highlight strategies developed by the districts, 
emphasizing the role of the state accountability 
system as the catalyst for change. 

The City Schools compilation cites a number of 
urban school districts in Texas and elsewhere that 
have improved academic indicators for minority 
students. These indicators range from moving 
students up to basic levels of performance (such as 
passing the TAAS), to earning higher-level diplomas, 
to reading at or above grade level. El Paso cut by 
half the gap in TAAS passing rates between African 
American and white students and Fort Worth 
reduced the passing rate gap between Latino and 
white students by 42%. The El Paso school did not 
adopt any special policy, while the Fort Worth 
school adopted a series of strategies with emphasis 
on professional development and support. 

Memphis schools doubled the number of African 
American students earning honors diplomas after 
the district eliminated lower level courses in the 
curricula. Boston schools increased the percentage 
of African American students scoring at the basic 
levels in the Stanford-9 tests after raising academic 
standards became a priority for the district in every 
subject and every grade. Charlotte schools also 
increased the percentage of African American 
students reading at or above grade level after the 
district adopted achievement goals to reduce 
disparities in academic achievement. 
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Recognizing that their African American students 
were lagging academically. Chapel Hill-Carrboro 
City Schools ( CHCCS) formed a “Blue Ribbon 
Panel” to analyze the problem and propose 
recommendations. Composed of school 
administrators, teachers, parents, and students, the 
panel presented a multifaceted plan that 
incorporated nationally known programs like 
Reading Recovery and AVID, and homegrown 
solutions like “Sister-to-Sister” (a mentoring 
program that pairs minority women in the nearby 
medical school with African American female high 
school students). The results for the African 
American students were mixed, with large 
increases in the mathematics scores, but lower 
increases, and even some decreases, in the 
writing assessments. After one year, the 
academic gap between African American and 
white students was still noteworthy. For instance, 
93% of the white tenth graders achieved 
proficient reading scores on the state test, 
compared to only 43% of their African American 
peers. However, the reform promoted a four-fold 
increase in the percentage of African American 
students in Gifted and Talented programs (from 
2% to 8%). 

Analysis 

As was found in the previous category, the data 
indicate that minority students start from a position 
of serious academic disadvantage and must walk a 
long path before they can reach basic levels of 
competency. The accountability movement has 
pushed these differences to the front stage. The 
condition of public schools, particularly funding 
differentials that lead to large class sizes, low 
teacher pay, lack of support or unprepared teachers 
may explain some of the large gaps between 
students at the high end (generally white middle 
class) and the low end (generally low-income 
minority) of the achievement spectrum. 

It is also possible that even at very early ages, 
society creates stereotypes about students who can 
succeed and those who cannot, and future 
interactions in school will be based upon these 



stereotypes. As indicated in Chicanos in Higher 
Education , which reports on interviews with 50 
Mexican American professionals with MD, PhD or 
JD degrees, poor minority students do not fit the 
idealized image of the successful, college-bound 
student. Teachers and counselors often tell these 
students that they cannot succeed and should not 
take challenging courses or apply to challenging 
schools. It took a highly focused and publicized 
reform for Chapel Hill teachers to find a “new 
group” of African American students able to attend 
Gifted and Talented classes, when these students 
had probably been ready for such a program for 
many years. Teacher preparatory schools should 
seriously examine their role in helping teachers to 
overcome such stereotyping behavior. 

The fact that schools across the country are raising 
the scores of students, including minority students, 
on different tests is commendable. It brings the 
hope that someday achievement gaps based on race 
or ethnicity will be only a memory. However, in 
fairness to the children, a note of caution must be 
sounded. By relying solely on test scores, these 
evaluations and reports miss other indicators that 
provide important information on academic 
achievement, including: dropout, expulsion and 
retention rates; referral to special education; and 
curriculum changes that may be occurring due to an 
emphasis on tests, such as the elimination of “non- 
testable” subjects (like music and art), or an 
emphasis on “testable” subjects at the expense of 
broader content. Moreover, since no follow-up of 
graduates is included (except for the Alaska reform), 
nothing is known about what happens with students 
from these districts or states after they leave K-12 
schools. Texas, for instance, despite its success 
with TAAS, ranks 34 th among 50 states and the 
District of Columbia for the percentage of graduates 
who go immediately on to two- or four-year 
colleges, and 45 th on the percentage of graduates 
who enroll in college within four years of 
graduation. 18 The debate on how to measure 
student achievement and the type of education that 
the country needs are essential components of the 
discussion on improving the academic achievement 
of minority youth. 
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High Sclhool/Transition Programs 

Overview 

This category reflects a mix of whole school 
programs and add-on interventions that propose to 
facilitate college access for students under- 
represented in postsecondary institutions, that is, 
low-income, minority students and students with 
“average” academic performance. Three are four- 
year, school-based programs geared toward students 
with a grade point average of C or better and who 
are motivated to pursue postsecondary education. 
All three programs have large proportions (80% or 
more) of minority students and require students to 
enroll in academically demanding courses. 

AVID is a nationwide program that targets C- 
average students who would be the first in their 
families to attend college. AVID offers one-on-one 
tutoring by college students, workshop classes on 
study skills and other supports. High School 
Puente aims to increase Latino participation in 
higher education by raising student skills and 
aspirations through critical thinking and writing 
assignments, college counseling and mentoring. 
Gateway to Higher Education is a New York City 
program with an emphasis on careers in science, 
medicine, and technology serving 95% minority 
students. To enter Gateway, students must score at 
least at the 50th percentile on New York City’s 
math and reading tests, have regular attendance, and 
GPA’s of 80 or better (on a 100-point scale). 
Summer and Saturday enrichment programs, 
tutoring and internships are some of Gateway's 
strategies. 

Three other evaluations describe high school 
programs with a college focus but do not mention 
selection criteria: Dare to Dream, Equity 2000 and 
GE Fund College Bound. Dare to Dream includes 
3 projects that propose a greater role for school 
counselors in keeping postsecondary options open 
for all students, including those who are considered 
at high-risk for school failure. The schools involved 
in the project were located in poor neighborhoods, 
with large proportions of minority students, and low 
levels of academic achievement. Equity 2000 is a 
whole school reform that requires all students to 
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take advanced mathematics courses while in high 
school. The program provides extra support to 
students through voluntary Saturday math 
academies and summer math programs. Minorities 
make up 72% of Equity 2000 participants. Like 
Equity 2000, the GE Fund College Bound provides 
block grants to schools and communities to institute 
programs that increase college access. Unlike 
Equity 2000, however, the GE Fund College Bound 
allows for greater flexibility in the strategies used by 
the grantee schools. 

Career Academies is the only representative in this 
report of high school programs dedicated to 
preparing students for fulfilling careers that are not 
necessarily dependent on a college degree. Career 
Academies are schools-within-schools that offer 
students an integrated academic and occupational 
curriculum and work-based learning experiences. 
More than 50% of the students in the Academies 
studied were Latinos arid 84% had GPAs of 2.1 and 
above. 

The evaluation of I Have a Dream (IHAD) includes 
two Chicago programs, one predominantly Latino 
and another 100% African American. IHAD 
connects low-income, inner city public school sixth 
graders with wealthy sponsors who provide 
mentorship and supports to help the youth pursue 
postsecondary education. The program offers long- 
term relationships from sixth grade until high school 
graduation, or even longer. Like IHAD, 
Philadelphia’s Sponsor-A-Scholar program provides 
academic supports to economically or academically 
disadvantaged high school students with B orC 
average grades who want to attend college. In this 
evaluation, 93% of participants were minorities of 
whom 76% were African American. The program 
matches these youth with trained mentors who 
accompany them from ninth grade through the 
freshman college year. Both IHAD and Sponsor-A- 
Scholar offer financial help to participants to defray 
tuition costs. 

Upward Bound operates parallel to the regular four- 
year high school, with students participating in after- 
school and Saturday classes often on college 
campuses. Upward Bound is the oldest of a set of 
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TRIO initiatives established by the Higher 
Education Act of 1965. TRIO programs aim at 
helping low-income and first generation college 
students enter and successfully complete 
postsecondary education. This national evaluation 
covers 67 sites with approximately 1,500 
participants. Nationwide, half of the Upward 
Bound participants are African American, 22% are 
Latino and 21 % white. 

Analysis 

Unlike the elementary school program evaluations, 
and despite the increasing numbers of high schools 
requiring standardized tests for graduation, the high 
school evaluations had little emphasis on test 
scores. 19 The broader range of academic 
achievement measures examined offers a better 
perspective of what is actually happening with 
students. Most documents include data on high 
school graduation and credits taken. A few have 
data on college entrance tests, such as the SAT and 
AP, and dropout rates. Follow-up is mostly limited 
to college enrollment, but GE Fund College Bound 
has data on college retention and Gateway for 
Higher Education collects college graduation 
information. 

All three programs that indicate some type of 
selection criteria for admission show good results, 
suggesting that a large group of C average students 
are ready to move up the academic ladder if 
provided adequate supports. AVID students 
maintain an average GPA of 2.94 and a 95% college 
enrollment rate. African American, Asian and Latino 
AVID students have disproportionately high 
enrollment rates in the California State and the 
University of California systems. High School 
Puente students, in relation to a matched 
comparison group, were more likely to take college 
entrance tests (SAT, ACT), complete more high 
school credits, and attend college, particularly four- 
year colleges, although no statistical differences 
were found in dropout rates and grade point average 
(GPA). The lack of difference in grades may reflect 
the fact that Puente students attend more 
academically demanding courses than the control 
group. African American students in Gateway are 
more likely to take chemistry and physics in high 



school than African American high school graduates 
nationwide. They are also more likely to have 
higher SAT scores. A 1996 survey with 330 
Gateway alumni revealed that 74% had graduated or 
would graduate from four-year colleges or 
universities within five years and 59% had remained 
in a science-related major or profession. 

The majority of programs featured in this report did 
not include cost data, but cost information was 
available for these three programs. The annual cost 
per student for Gateway in 1997 was $1,600 above 
the mean per pupil expenditure in New York City. 
The state’s annual per pupil expenditure for High 
School Puente was $480, but training costs were 
partially subsidized by the University of California. 
The average cost of AVID for schools and districts 
in Year One per student per year outside of 
California is $540 (about $3 per day). By year 
three, the cost drops, on average, to under a dollar 
per student per day. In California, where AVID is a 
state-supported program with 1 1 regional centers, 
the average cost of AVID for schools and districts is 
about $180 per student per year. 

Of the programs that do not indicate admission 
criteria, most Dare to Dream high schools doubled 
the enrollment of African American and/or Latino 
students in Advanced Placement and college 
preparatory courses (the report did not publish 
passing rates). Districts adopting the Equity 2000 
program also showed increased enrollment of 
minority students in college gateway courses. 
However, passing rates in these courses did not 
increase accordingly. The number of students 
taking college entrance exams (SAT, ACT) 
increased in all GE Fund College Bound schools 
after five years, but the program had little impact on 
test scores, high school graduation rates, or dropout 
rates. When compared to a national database, GE 
Fund students, particularly Latino students, had 
higher college enrollment and retention rates. The 
evaluation of Career Academies found statistically 
significant improvements for students who had 
entered the program with high risk of school failure 
but not for those in the middle and low risk 
categories (see the summary’s methodology for an 
explanation of the risk categories). 
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Evaluators of I Have a Dream used a matched 
group of students taken from other sixth grade 
classes in the same schools as IHAD participants. 
Participants were twice as likely to graduate from 
high school and three times more likely to enroll in 
college than the comparison group. The Sponsor-A- 
Scholar evaluation examines a sample of high 
school graduates from 1993 through 1997. The 
sample was divided into matched sub-groups of 
program participants and non-participants and 
compared in terms of GPAs and college enrollment 
one year and two years after high school graduation. 
In general, program students had statistically 
significant higher GPA and enrollment rates than 
non-participants. Gains were higher for students 
who started the programs with lower grades, stayed 
in the program longer, and met more frequently with 
their mentors. 

For Upward Bound, program participants were 
compared to a matched control group. Differences 
between the two groups were not statistically 
significant for average GPA and enrollment in 
postsecondary institutions (including vocational/ 
technical schools). Latino and white participants 
earned two more high school credits than peers in 
the control group while African Americans earned 
more Advanced Placement credits. Results were 
correlated to time in the program and expectation 
about attending college at the onset of the program. 
The longer the student remained in the program and 
the lower the initial expectation, the stronger the 
results. However, more than 55% of the 
participants left the program before high school 
graduation, a finding evaluators attributed to 
students’ needs for paid employment competing 
with Upward Bound’s after-school and Saturday 
classes. 

Overall, programs that provide extra attention and 
supports to high school students, particularly those 
who average C or better, are succeeding in moving 
them to postsecondary education. The majority of 
the evaluations do not describe what happens when 
the students get to the next level. However, a few 
do: GE Fund College Bound students have higher 
college retention rates; a small group of Gateway 
students show high college graduation rates; and 



Upward Bound students are less likely to need 
remedial classes while in college. 

Posttsecomdary School Programs 

Despite the intensive search for evaluations of 
postsecondary programs that serve minority 
students and disaggregated the data, few studies 
were found and most of them were not evaluations, 
but descriptive reports. At the beginning of the 
search, we contacted a large number of 
organizations that provide college scholarships for 
minority youth. None had evaluations. We 
received suggestions and indications about “great 
studies” being done in one state or another, only to 
find that these studies would not meet the 
acceptance criteria for rigorous evaluations 
disaggregated by race or ethnicity. The landscape 
of evaluations of postsecondary interventions for 
minority students with disaggregated data is as arid 
as the programs are numerous. 20 

The six postsecondary reports are examples of the 
variety of programs that are being implemented at 
the postsecondary level to help minority students 
break the barrier of the K-12 years and enter higher 
education. Three summaries describe programs that 
support minority students at different points along 
the journey through college and graduate school: 
the Emerging Scholars Program (ESP) helps 
undergraduates to remain in college; the Puerto 
Rico Louis Stokes Alliance for Minority 
Participation (PR-LSAMP) offers support for 
women and minorities pursuing graduate degrees; 
and Compact for Faculty Diversity provides a 
bridge for minority students as they complete their 
doctorates and enter college or university teaching 
positions. While Compact does not emphasize a 
particular specialization or field, both ESP and PR- 
LSAMP focus on the fields of mathematics, sciences 
and engineering, where minorities have been 
traditionally under-represented. Two studies focusing 
on Historically Black Colleges and Universities and 
Tribal Colleges offer descriptive data on the role of 
these institutions in the lives of African Americans and 
Native Americans. Chicanos in Higher Education is 
an example of a number of qualitative studies that 
provide a voice to minority individuals and shed 
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some light on factors that influence their 
professional success. 

The Emerging Scholars Program (ESP) was 
initially developed at the University of California, in 
the 1970s, to improve the retention and success of 
minority students who enter mathematics-related 
majors. Currently operating under several different 
names in over 100 universities across the country, 
the basic ESP model utilizes extended discussion 
seminars and small study groups to help students 
succeed in the calculus course sequence at the 
beginning of their majors. With additional professor 
and peer support, these students form small learning 
communities that work as teams. Evaluations in 
Texas and Wisconsin revealed that ESP students 
were two to five times more likely to get As and Bs 
in calculus than their peers outside the program. A 
study at the California Polytechnic Institute showed 
that only 15% of ESP students had changed majors 
or left college within three years, compared with 
52% of the students in a control group. They were 
also more likely to complete their mathematics 
requirement one academic quarter earlier than the 
control group. 

The Puerto Rico Louis Stokes Alliance for 
Minority Participation ( PR-LSAMP ), funded by 
the National Science Foundation and the federal 
government, has a dual goal: to foster the 
involvement of women and minorities in the fields 
of mathematics, science, and engineering; and to 
promote innovative teaching strategies that improve 
students’ performance in those fields. According to 
the report, of all bachelor’s degrees in science, math 
and engineering earned by Latinos in the U.S. in 
1997, 25% went to PR-LSAMP students. From 
1993 to 1998, PR-LSAMP students earned 11% of 
engineering PhDs and 17% of natural science PhDs 
received by Latinos nationwide. 

Compact for Faculty Diversity is a consortium of 
regional education organizations and universities that 
provides financial support and a peer network for 
minority graduate students. The Compact's annual 
Institute for Teaching and Mentoring brings together 
minority graduate students and professors from 
across the country to discuss possibilities and pitfalls 



in the world of higher education. Of the 435 
scholars served by the program, 92% had 
completed or were continuing their degrees. Of the 
Compact alumni who had earned a PhD, 70% were 
in tenure-track faculty positions and 18% were in 
post-doctoral positions. As with the ESP model, 
Compact promotes a small supportive community of 
peers and professionals that guides the graduate 
students into careers in higher education. 

Historically minority-serving institutions continue to 
play a crucial role in minority higher education, and 
this report includes studies of Historically Black 
Colleges and Universities (HBCUs) and Tribal 
Colleges } x The study of HBCUs shows that 
HBCUs graduate more African American students 
than other institutions. In the mid-1990s, 21% of all 
African American undergraduates attended HBCUs , 
but 28% of African American graduates got their 
degrees from HBCUs and 33% of the African 
American college students taking the Graduate 
Record Examination came from HBCUs . 

Tribal Colleges' original purpose was to facilitate 
access to higher education for Native Americans 
living on reservations and to provide educational 
opportunities without forcing assimilation into 
mainstream white culture. Today, there are 33 
Tribal Colleges serving more than 10,000 Native 
American students. Tribal Colleges have an 
important role in diversifying faculty composition. 
Compared to other institutions of higher education 
that employ on average less than 1 % Native 
American faculty and staff, 30% of the faculty and 
70% of the staff at Tribal Colleges are Native 
Americans. 

Chicanos in Higher Education reports on 
interviews with 50 Mexican American professionals 
with MD, PhD or JD degrees. All came from low- 
income, immigrant families, composed mostly of 
farm workers and other unskilled laborers. Most 
began school with Spanish as their primary 
language, yet all completed a doctoral-level 
education from the country’s most prestigious 
institutions. The interviewees stressed the 
importance of supportive parents and a family 
environment that was conducive to learning. At 
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least half cited the presence of a caring adult outside 
the family who functioned as a mentor, encouraging 
and prodding them toward academic success. 
Though most of them eventually got into college 
preparatory programs in high school, they had to 
fight a system that insisted on tracking them into 
less demanding curricula. Latino recruitment 
programs, scholarships for high-achieving scholars, 
stipends for low-income students, and a lot of hard 
work complete the list of factors that, according to 
the interviewees, contributed to their success. 

Conclusion 

All the selected evaluations of early childhood 
programs included follow-up, and some of them for 
substantial time periods. They also provided a 
variety of data to indicate that these programs are 
attaining their objectives of providing low-income 
children, including minority children, with more 
resources to succeed in later years. The evaluations 
of K-8 programs and district or statewide school 
initiatives have a limited focus on test scores. They 
tell us that many schools and states are raising the 
scores of minority students on different tests or are 
raising their passing rates in these tests. What this 
represents for the children’s future is not clear. 



However, it must be emphasized that, at least the 
programs and initiatives are raising these children’s 
scores and passing rates. Doing nothing would be 
much worse. Rather than being a criticism of the 
existing data, this comment represents a longing for 
more data. 

The evaluations of the high school programs 
diversify their measures. Although little is said 
about students’ test scores, the information indicates 
that minority students in those programs are, in 
general, graduating from high school and going to 
college in greater numbers. The summaries on 
postsecondary education end this chapter with a 
message of hope, showing minorities who are 
succeeding in demanding careers, such as those 
related to sciences, mathematics and technology, 
and attaining faculty positions in universities. This 
message cannot be missed, because, as the 
summary on Chicanos in Higher Education 
suggests, many of those successful youth start their 
school years in the “high risk” category. On the 
whole, the summaries in this report highlight the fact 
that no student should be discounted as a lost cause. 
The opportunities and supports necessary to achieve 
success at the highest levels of our educational 
system must be available to all. 
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Chapt@r3: The Search f@r 

“Magic Bullet 91 



What makes programs successful? What do they 
offer so that young people challenge themselves and 
succeed? These questions are at the core of this 
report as they were in AYPF’s two previous 
compendia and of a more recent AYPF report. 
Raising Academic Achievement: A Study of 20 
Successful Programs. 12 That report identified five 
overarching strategies shared by programs that 
raised academic achievement: 

♦ High standards for participants, programs and 
staff, including strategies that ensured the 
quality of implementation, and demanded high 
performance from youth and staff alike. 

♦ Personalized attention , that is, strategies that 
enable the staff to know the program 
participants as individuals, with unique needs, 
strengths and weaknesses. 

♦ Innovative structures where the needs of the 
students, rather than tradition or bureaucracy, 
guide the teaching/leaming process. 

♦ Experiential learning , bridging school and 
society. 

♦ Long-term support that gave youth time to 
create trust and develop stable relationships and 
supports. 

For this report, the same analytical process was 
used. A matrix was created with the program 
components and “contributing factors” as cited in 
the evaluations and program literature. Through a 
process of coding, the differences and similarities 
among components were highlighted and/or 
aggregated into categories, until a group of ten 
strategies remained that were shared by nine or 
more of the programs. In contrast to the previous 



reports that focused solely on “successful” 
programs, this report includes programs that have 
both positive and negative findings. We include all 
evaluations in the analysis, regardless of findings, in 
an attempt to understand whether good results can 
be attributed to specific components, to a specific 
mix of components, or to some other variable that 
merits further investigation. In the case of reports 
describing different programs, the shared strategies 
cited in the documents were also included. 

The overarching strategies found in Raising 
Academic Achievement are again reflected in this 
report except for “Experiential Learning,” which is 
cited only once in connection with the Career 
Academies summary. “High Standards” is 
represented here by the three most frequent 
components: program quality, academically 
demanding curricula and professional development. 
“Personalized Attention” is discussed in two 
contexts: school strategies to reduce the student-to- 
teacher ratio and strategies to provide youth with 
extra, individualized supports. The two remaining 
overarching strategies, “Innovative Structures” and 
“Long-term Supports,” are also represented. 

This chapter discusses the strategies used by 
programs whose evaluations we have summarized. 
However, a few words of caution must be shared. 
First, no “magic bullet ” was found , that is , no one 
strategy is common to all programs that have 
good findings . Second, the sample is limited to 38 
reports, several of which have less than stellar 
evaluations. Therefore, these findings should be 
considered as guidelines for further inquiry rather 
than prescriptions for success. 

The ten most frequent strategies identified in this 
report are listed below from most to least frequently 
cited in the program evaluations: 
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♦ Program quality 

♦ Academically demanding curriculum 

♦ Professional development 

♦ Family involvement 

♦ Reduced student-to-teacher ratios 

♦ Individualized supports 

♦ Extended learning time 

♦ Community involvement 

♦ Long-term supports for youth 

♦ Scholarships and/or financial support 

Program Qualify 

“High standards” is a catch phrase in today’s 
education policy debates. Virtually all programs 
affirm their commitment to “high standards” 
without defining the concept. AYPF’s perspective is 
that high standards must include a concern with the 
quality of the program and its staff before demands 
are made from participants. Quality of 
implementation, leadership and accountability are 
three essential strategies that help ensure high 
program standards. 

Quality of implementation 
The quality of implementation is demonstrated 
through careful planning and timely and efficient 
resources that are targeted to specific goals. For a 
school to receive certification as an AVID center, it 
must fulfill a series of requisites, including training 
for the site administrator, program coordinator, 
teachers and tutors; identification of resources for 
implementation and sustainability; selection of 
students; and integration between the program and 
the regular school day. When the Calvert Program 
was introduced at the Woodson School, a full-time 
coordinator was hired to oversee implementation of 
the program and its daily operations. GE Fund 
College Bound stresses the substantial size and 
long-term support of its GE Fund grants — at least 
$250,000 for five years — as giving adequate time 
and resources to plan and implement the reforms 
necessary to improve school-wide academic 
achievement. Evaluators note that one reason that 
Class Size Reduction in California may not have 
shown minority academic achievement gains was 
that the program did not include timely and 



sufficient resources for successful implementation in 
high-poverty, predominantly-minority schools. 

Leadership 

Leadership is essential to ensure program quality 
and sustainability. As charter school principals, the 
directors of KIPP Academies have complete control 
over budget and personnel decisions, thus allowing 
them to be better leaders at the school level. KIPP 
principals lead by example. In addition to being 
administrators, they are teachers who do not 
hesitate to step out of their offices and into the 
classroom to do the nitty gritty work of education. 
With the help of Gap, Inc., they have also started a 
fellowship program that will train a corps of 
educational leaders to found their own charter 
schools across the country serving disadvantaged 
youth. Gateway for Higher Education has had the 
same co-directors since its founding in 1986 and, 
according to the evaluators, this continuity has 
contributed to the program’s strong sense of 
purpose. Dare to Dream and the Alaska Onward to 
Excellence/Alaska Rural Systemic Initiative are 
based on the concept of shared leadership. In the 
projects described in Dare to Dream , school 
administrators, teachers, school staff, outside expert 
teams and students work together to find solutions 
for existing problems or to propose new options. 

The Alaska project relies on a sense of shared 
ownership between program staff and community. 
GE Fund College Bound describes some of their 
program efforts as being enhanced by strong 
leadership exhibited by the schools and their GE 
partners, while others were hampered by frequent 
leadership turnover or weak support from principals 
and school administrators. 

Accountability 

Public school “accountability” is a growing concern 
of local, state and federal governments. Tests, 
particularly state-developed tests and school report 
cards, are tools commonly used to provide 
stakeholders with feedback about the performance 
of their schools and students. In many states, 
schools that do not attain some pre-established 
benchmark on the state tests are threatened with 
sanctions. The accountability movement has been 
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particularly well documented in Texas. The Texas 
Assessment of Academic Skills (TAAS) is a 
criterion-referenced test administered annually that 
measures student achievement in reading and math 
(for grades 3-8 and 12), in writing (for grades 4, 8, 
and 12), and in science and social studies (grade 8). 
Texas students must pass the TAAS to graduate 
from high school. The evaluations of programs 
using TAAS data show schools increasing the 
percentage of minority students passing the TAAS 
by 40% or more, while in many other states, test' 
score improvements for minorities are incremental. 
Yet successful school programs in Texas are quite 
varied. In fact, the evaluation of four school districts 
{Texas District Wide Initiatives ) attributes their 
success to the politically-imposed accountability system 
rather than to specific strategies. 

It appears that the political climate favoring 
accountability has positive facets that must be 
considered. As states begin to require all children to 
perform, even those labeled “at risk,” educators 
must pay attention to all children, defining clear 
expectations for all, and find ways to help those 
who are failing to achieve academically. It is 
important to observe that accountability should not 
be confused with high standards, since many states, 
including Texas, still rely on minimum competency 
tests. How to fairly and equitably use the 
advantages of accountability for minority academic 
achievement gains is still an open question. 

Academically Demanding Curricula 

All early childhood programs included in this report 
provide pre-school aged children with challenging 
educational activities that are also developmental^ 
appropriate. Abecedarian ’s curriculum includes 
arts, language, and literacy, in addition to fine motor 
skills development. The Child-Parent Center 
curricula emphasize language and mathematics 
through a variety of learning experiences. Head 
Start programs have incorporated academic 
activities with their full-service mission. High/ 
Scope Perry Preschool offers a well-structured 
curriculum with emphasis on language, literacy, 
music and mathematics. 



Concern with challenging curricula was equally 
apparent in K-12 programs. The Calvert Program 
emphasizes reading comprehension and required 
weekly compositions even for first graders. Since 
Memphis City Schools (described in the City 
Schools report) eliminated lower level courses in 
high schools, the percentage of African Americans 
graduating with an honors diploma doubled. The 
Alaska Onward to Excellence/Alaska Rural 
Systemic Initiative is based on a dual commitment 
to equity and excellence. Eskimo and Native 
American children from rural villages are taking 
college-entrance tests and going to college in higher 
numbers since the introduction of the program. All 
Gateway for Higher Education students are 
expected to complete a minimum of three Advanced 
Placement courses. 

However, the requirement to attend academically 
demanding courses must be accompanied by 
appropriate supports. The low algebra passing rates 
for students in Equity 2000 may have been due to 
the limited supports for students enrolled in algebra 
and lack of support relating to other types of high 
school coursework or college attendance. The 
persistent gap between minority and white students 
in High Schools That Work , which eliminates 
general education and sets high standards for all, 
indicates a need for additional supports geared 
toward these students. 

Professional Development 

To maintain the quality of any program, it is not 
enough to create mechanisms for quality control. 
Staff must be prepared to respond to the challenge. 
Gateway , which creates a school-within-a-school for 
academically talented students, carefully selects its 
teachers based on their background, experience, and 
dedication. For other programs, particularly those 
with less ability to select staff, professional 
development and training is an important program 
component. 

AVID and High/Scope require staff training before 
implementation. Success for All provides a three- 
day summer training session and continued on-site 
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staff training during the year. Project GRAD offers 
training and ongoing support for teachers to reduce 
turnover. According to the evaluator, as the 
facilitators operate outside the teachers’ assessment 
process, teachers feel comfortable asking for help 
with classroom problems. The schools described in 
Urban Elementary Schools introduce professional 
development activities at the time when changes in 
curriculum or school structure are implemented. 
Programs that rely on tutors or mentors, such as 
High School Puente , 1 Have a Dream and Sponsor - 
A-Scholar, offer them training and supervision. At 
the Boys and Girls Clubs of America, trained staff 
provides support to volunteers. 

However, as the CAPE evaluators observed, 
offering professional development does not 
necessarily ensure that teachers will profit from it. 
CAPE offered extensive professional development 
to teachers and artists, including nearly a dozen 
workshops throughout the year. Yet, most 
participants attended no more than three workshops 
due to lack of time. 

Family Involvement 

Approximately 40% of the selected evaluations 
report activities geared toward improving 
communication with families, or increasing family 
involvement with the programs. Although such 
efforts are concentrated on initiatives for young 
children, at least two high school programs also include 
activities to promote greater involvement of families. 

Early childhood programs focus on helping parents 
provide adequate support for their child’s 
development. Therefore, these programs offer a 
range of activities that include family education, 
advocacy and support. Information on childhood 
development, health and nutrition is provided in all 
the programs, either through workshops or home 
visits. In High/Scope, families and staff met 
monthly to discuss developmental issues. Program 
staff also made weekly home visits to families, 
meeting with the child and the family to model 
classroom activities. Abecedarian and Head Start 
used home visits with the objective of information 
and support and involve families on advisory boards 



and committees with planning and managerial 
functions. 

The two after- school programs in this report, Boys 
& Girls Clubs of America and Sacramento START, 
include families in their activities, generally as 
volunteers, although S7A/?rhires participants’ 
families to staff the program. Among the K- 1 2 
programs, AVID , Alaska Onward to Excellence/ 
Alaska Rural Systemic Initiative , Calvert Program , 
Chapel Hill-Carrboro City Schools, High School 
Puente and Project GRAD have family involvement 
components. Of these, AVID and High School 
Puente are exclusively for high school-aged youth, 
and Calvert is an elementary school program. The 
remaining initiatives serve K-12 students. 

AVID emphasizes communication between families 
and the program, offers workshops on the college 
application process, and includes families on its 
advisory board. Local families and community 
members are also part of advisory boards in the 
schools involved with the Alaska reforms that 
encompass grades K to 12. Another K-12 initiative 
in the Chapel Hill-Carrboro City Schools includes 
families as volunteers and members of the Blue 
Ribbon panel that designed the reform. Teachers 
and administrators are encouraged to set up 
meetings in families’ homes or workplaces to 
facilitate participation. In addition, the schools offer 
assistance for students who are parents with a 
Young Parent Institute and the Adolescent Parenting 
Program that provide monthly support groups and 
infant health education. Project GRAD implements 
a comprehensive family outreach program that 
includes activities to recruit students and their 
parents. During its community-wide Walk for 
Success, alumni, teachers, staff, mentors, university 
volunteers, and community leaders go door to 
door to over 1,600 families per year to raise 
awareness of the program. Project GRAD also 
has Parent Universities to improve parental 
literacy and involvement. In addition, alongside 
teachers, principals and other community 
members. Project GRAD families participate in 
decision-making committees that manage the 
project’s feeder schools (elementary and junior 
high schools). 
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Reduced Student-to-Teacher Ratios 

Many selected programs that show academic gains 
for minority students include strategies to reduce 
student-to-teacher ratios. Tennessee’s Project 
STAR and Wisconsin’s SAGE are statewide 
experiments with reduced class sizes for grades one 
through three. STAR used classes with 17 students 
per teacher and SAGEs classes averaged 15 
students per teacher, compared with traditional 
classes of 20 to 25 students. Participants in the 
small classes, particularly African Americans, had 
higher average test scores than students in the larger 
classes. STAR did not find gains when two teachers 
or a teacher and teacher’s aide were in the 
classroom, but SAGE found similar gains in 
strategies that reduced student-to-teacher ratio by 
increasing the number of teachers in regular 
classrooms, including team teaching and floating 
teachers. In California, as described in Class Size 
Reduction , classes were reduced from 30 to 20 
students or less. Different from STAR and SAGE , 
two carefully designed and implemented reforms 
that began as pilots, the Californian project was 
imposed statewide and, particularly in those in low- 
income areas, the class reduction occurred at the 
expense of other resources, such as music 
instruction and school libraries. 

Rather than reducing the number of children per 
classroom, Child-Parent Centers increase the 
number of staff, placing two teachers for each 
classroom of 17 toddlers or 25 kindergarteners. 

The Chicago Arts Partnership in Education uses 
the co-teaching system, with a teacher and an artist 
working together to integrate arts into the academic 
curriculum. Career Academies are schools- within- 
schools that provide self-contained environments 
within larger institutions where students have closer 
interaction with staff. KIPP Academies are small 
charter schools with no more than 300 to 400 
students. High School Puente selects about 30 
students per cohort. Success for All uses small 
reading groups divided by literacy level, rather than 
age; this program is also a component of Project 
GRAD. The Urban Elementary Schools report 
indicates that some schools also reduced the number 
of students per class. 



Cost is an issue in projects that demand expansion 
in buildings and/or personnel, but only one of the 
studies ( Child-Parent Centers) included a cost 
benefit analysis. It is important to observe that 
other programs such as those described in Texas 
District Wide Initiatives and City Schools that 
show significant academic gains, particularly for 
minority students, do not report the use of smaller 
classes or small learning environments. 

Individualized Supports 

For students who are struggling academically, 
individualized support may be the difference 
between falling behind and moving ahead. In 
addition to the involvement of the students’ 
families, many programs utilize community 
members, college students, employers and other 
groups as tutors and mentors to address the 
academic needs of specific students, or offer 
support, feedback and encouragement. Tutors or 
mentors can also function as role models, guiding 
the youth through difficult transitions and into a 
college and/or career path. 

Tutors and mentors are found at all levels of the 
educational ladder. For instance, Success for All, 
a program for elementary school children, uses 
trained tutors to help students in need. Minority 
students from the University of North Carolina 
provide tutoring for elementary, middle, and high 
school youth at Chapel Hill-Carrboro City 
Schools. Their “Sister to Sister” program pairs 
African American females in medical school with 
ninth grade “sisters” for support and role 
modeling. At the undergraduate level, Emerging 
Scholars pairs a teaching assistant with one to 
two undergraduate students to tutor calculus. 
Compact for Faculty Diversity organizes an 
annual institute where university and college 
professors share their experiences with PhD 
candidates and mentor them through the process 
of moving from graduate students to faculty 
members. 

The use of tutors and mentors is frequent among 
high school programs as well. AVID uses college 
students to provide one-on-one tutoring to C- 
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average high school students who dream of 
entering college. Employers are actively involved 
in Career Academies, sitting on boards, helping 
with curriculum planning, and also mentoring 
students in work-based experiences. Gateway to 
Higher Education offers after-school tutoring 
programs. At High School Puente , “peer 
partners” help the students to transition from 
middle to high school. In addition, adult mentors 
work with the students throughout high school. A 
Community Mentor Liaison (CML) is dedicated 
to recruiting, training and matching the mentors 
with the students. GE employees tutor students 
at GE Fund College Bound schools, offering 
homework assistance and other supports. Tutors 
are also procured among community volunteers. 

1 Have a Dream , Sponsor-A-Scholar and Upward 
Bound all use mentors. The mentors in I Have a 
Dream and Sponsor-A-Scholar are intensely 
involved with the students, monitoring their 
academic performance, providing opportunities 
for recreational activities, and internships, and 
offering financial support through college. 

Using tutors and mentors is a less expensive strategy 
to reduce the student-to-adult ratio than using 
certified teachers, but it is also a riskier strategy. 
Unqualified, untrained and unsupervised tutors or 
mentors can sometimes do more harm than good. 

Extended Learning Time 

Some programs use longer school hours, extra 
school days, Saturdays and summer courses to 
provide students with more learning time. For 
preschool aged children, any formal instruction time 
may be considered extra time, and that is offered by 
all early childhood programs in this report. 
Abecedarian functioned 8 hours a day, 5 days a 
week for 50 weeks. Child-Parent Center preschool 
programs are offered for 3 hours in the morning or 
in the afternoon, and kindergarten programs are 
either half day or full day. High/Scope Perry 
Preschool had 12 V 2 hours of instruction per week. 

Boys and Girls Clubs of America (B&GCA) and 
Sacramento START are after-school programs that 
provide low-income, mostly minority children with 



extra educational supports. B&GCAs are open 5 to 
6 days a week, 6 to 7 hours a day. In addition to 
recreational and social activities, some clubs offer 
an educational program that includes homework 
support, structured discussions on educational 
topics, 1 to 2 hours a week of writing, 4 to 5 hours 
per week of reading, and additional time for 
educational games, such as word and math games. 
For middle to high school students, the clubs also 
offer technology training and career exploration 
programs. Sacramento START functions 9 hours a 
week and also includes homework assistance, 
literacy training and other educational activities. 

The program staff maintains ongoing 
communication with the schools to align curricula 
and learning goals for their participants. Children in 
both programs show academic gains. 

Among the school programs that offer extra- time, 
Gateway functions for 1 1 months a year and Project 
GRAD offers after-school programs. Emerging 
Scholars and Equity 2000 have Saturday and 
summer activities, although attendance is voluntary 
in Equity 2000. The activities in I Have a Dream , 
Sponsor-A-Scholar and Upward Bound are all an 
added value to the regular school day. The KIPP 
motto is that “there are no shortcuts,” and the time 
commitment of students and teachers exemplifies 
this philosophy. Students attend class from 7:30 
AM until 5:00 PM Monday through Thursday and 
until 4:00 PM on Fridays. They spend four hours 
at the school on most Saturdays and attend 
additional courses four weeks every summer. 

Community involvement 

Alaska Onward to Excellence/Alaska Rural 
Systemic Initiative and Chapel Hill-Carrboro City 
Schools ( CHCCS) highlight the power of 
communities to promote and support school 
changes. The Alaska reform was guided by 
community members upset with the state of their 
schools. Community participation is essential to the 
program, reinforcing cultural traditions and 
knowledge that are interwoven with the more 
traditional curricula. In North Carolina, community 
representatives sat on the Blue Ribbon panel that 
proposed the CHCCS strategies to improve the 
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academic achievement of African American 
students in the school district. 

CAPE represents an innovative way to involve 
artists and community organizations in schools to 
enhance education through arts. Career Academies 
involve the business community in planning and 
supporting the program, in addition to offering 
work-based opportunities for the students. Some of 
the Urban Elementary Schools also report business 
involvement, while Gateway has partnerships with 
museums and research centers to provide students 
with educational and internship opportunities. 

Long-Term Supports for Youth 

Several programs encourage long-term, stable 
relationships between participants and 
knowledgeable adults. Abecedarian , Child-Parent 
Centers and High/Scope are all five-year programs 
with long follow-up. Abecedarian also includes a 
summer program to help participants in their 
transition to public school. The mentor-youth 
relationship in I Have a Dream and Sponsor-A - 
Scholar remains for more than five years, and helps 
youth transition into postsecondary education. 
Evaluations of Sponsor-A-Scholar and Upward 
Bound found that the longer youth stay in the 
programs, the greater their academic gains. Since 
transitions are important periods in any person’s life, 
particularly for youth who have weak social 
supports, it is puzzling that so few of the programs 
reviewed offer extra supports during transition, 
particularly from middle to high school. 

SchoBairships and/or Financial Support 

Several K-12 programs offer financial help to 
students who demonstrate high academic 
performance. CHCCS offers scholarships to 
African American students who enroll in two- or 
four-year colleges. Scholarships are also provided 
in some GE Fund programs. I Have a Dream and 
Sponsor-A-Scholar supplement the costs of college 
that are not covered through other scholarships or 
loans. The voucher movement proposes 
scholarships to defray the costs of private school 
tuition for families whose children are in failing 



public schools. The summary of School Vouchers 
describes a three-city experiment. The report 
indicates that the scholarships did not cover the full 
tuition but does not explain how low-income 
families were able to cover the remaining costs, a 
requirement that may hamper the use of vouchers 
for families in the lowest income brackets. 

Among the postsecondary programs, only Emerging 
Scholars does not report financial aid. Chicanos in 
Higher Education, which interviewed Latinos who 
excelled professionally, cites the importance of 
minority recruitment programs, scholarships for 
high-achieving students, and stipends for low- 
income students as tools to break the cycle of 
poverty for low-income minority students who 
aspire to a college education. Compact for Faculty 
Diversity works with states and graduate institutions 
to ensure continuity of funding and supports for 
minority students as they complete their doctoral 
degrees and enter academic life. The Puerto Rico 
Louis Stokes Alliance for Minority Participation 
offers stipends for low-income students who excel 
academically and pays travel costs of students who 
participate in conferences. 

Conclusion 

These evaluations highlight programs that are 
succeeding in improving the academic achievement 
of African American, Latino and Native American 
students. Most programs are bringing minority 
students at the lowest level of academic 
performance to the minimum required level of 
competency for their grades, such as those 
described in Texas District Wide Initiatives. A few, 
like AVID, are helping students already at the middle 
to attain higher levels of achievement, while 
programs like Gateway improve the performance of 
students who are close to becoming high achievers. 
Evaluations such as those for the GE Fund and 
Upward Bound reinforce the value of investing in 
low achieving students, proving that they can profit 
from supportive interventions. 

What can be learned from this chapter? The first 
lesson is to intervene preventively, even before the 
child enters school, to avoid the gap between high 
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and low achievers. A concern with early 
intervention does not imply abandoning youth who 
are struggling academically in the remaining school 
years. The evaluation of Head Start shows a 
decline in academic gains as the child moves 
through grades, and so does the evaluation of 
Project STAR. Indeed, learning is a dynamic 
process that must be supported throughout the 
school years. 

The second lesson learned is that no one approach 
guarantees academic success, although a few 
strategies carry more promise than others. Highly 
structured programs, such as Calvert or Success for 
All , have successful outcomes, but so does a 
creative, flexible program such as CAPE. Overall, 
the summaries suggest that demanding high 
performance from programs, staff and students is 
essential for a successful program. Most programs 
that show positive results implement mechanisms to 
ensure program quality, maintain well-trained 
teachers and support staff, and provide 
academically demanding courses. 

A lesson from the less successful programs is that 
pushing youth who are already struggling 
academically into demanding courses without the 
necessary supports may simply create a wave of 
failures and frustration that will eventually drive the 
youth out of schools, rather than toward graduation. 
This finding, far from leading to the defeatist 
conclusion that these youth have no hope, should 
guide us to the question of “what needs to be done 
that these programs are not doing?” Strategies to 
support students are varied and many successful 
programs mix strategies to reduce the student-to- 
teacher ratio (such as reduced class sizes, small 
schools and team-teaching) with the presence of 



volunteers, tutors or mentors to ensure more 
individualized attention for all students. In addition, 
good programs provide high quality professional 
development for staff, tutors, and mentors. 

Financial support is essential for low-income 
students who dream of pursuing postsecondary 
education. Programs that encourage the 
participation of families and community 
representatives increase the support network and 
create a culture of academic achievement around 
the student. 

The evaluations summarized here also teach about 
the power of persistence. The Texas accountability 
system and the Alaska reforms are a decade old. 
Changes in education do not occur in a short period 
of time. Unfortunately, many reforms come and go 
abruptly, leaving educators without time to 
implement them adequately, and students without 
time to profit from them. 

One common denominator among the selected 
programs is a heightened level of attention toward 
all students in an attempt to reach benchmarks that 
were established by the school, district or state. 
Interviews with successful Mexican American 
professionals ( Chicanos in Higher Education) 
suggest that educators tend to give up on low- 
income, minority students who do not fit their 
idealized image of the successful, college-bound 
student. By disaggregating their data, school 
districts highlight inequalities within their system, a 
necessary step toward correcting them. A final 
lesson that may be taken from these evaluations is 
that commitment to all students, more than 
specific strategies, appears to prevail as the main 
contributing factor of success . 
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Chapter 4: Moving Forward 



Summary of Findings 

In this report, a detailed picture is presented of the 
available research on programs that have been 
found to improve minority academic achievement. 
Despite continuing achievement gaps, the youth 
programs and school initiatives included in this 
report provide concrete examples of efforts to 
increase achievement for minority youth. 

♦ Evaluation findings were particularly strong and 
positive at the early childhood level. When 
compared to control groups, minority children 
who attend early childhood development 
programs are more likely to remain in school, 
complete more years of education, and require 
less special education. These evaluations show 
a pattern of improvement that cannot be denied. 
The message from this body of evidence is that 
early childhood programs increase the chances 
for minority children to do well in school and in 
later life. However, no education system can be 
satisfied with good early intervention programs 
without strong K-12 schools that will maintain and 
expand the educational gains of the early years. 

♦ The elementary through middle school 
evaluations were almost exclusively focused on 
test scores. In most cases, improvements were 
incremental and even where minority academic 
achievement increased, the disparities in 
achievement between minority and white youth 
were highly apparent. Texas is probably the 
only state where achievement gaps between 
minorities and white students are being halved 
or cut even more. However, Texas students are 
measured on passing rates on only a minimum 
competency test. The question of whether 
higher levels of achievement are reached 
remains unanswered. 

♦ Because they focus on more than test scores, 
the high school/transition programs offer a 
better perspective of what is actually happening 
with their minority students. Among the 



positive findings of some of these programs 
were one or more of the following: increased 
high school graduation, more high school credits 
earned, higher GPAs earned or maintained, 
more college prep and Advanced Placement 
courses taken, increased enrollment in higher 
level mathematics and science classes, more 
college entrance exam-taking and higher scores, 
less need for remediation in college, higher 
levels of college enrollment at two- and four- 
year colleges, higher levels of college retention 
and graduation, and continuation in science- 
related majors or professions. 

♦ Fewer quality evaluations were available at the 
postsecondary level with data disaggregated by 
race or ethnicity. The postsecondary programs 
included in the report show African American, 
Latino and Native American youth succeeding 
in demanding careers and entering universities 
not just as students, but as professors as well. 
However, their numbers are still quite small. 

Recommendations 

Based on AYPF’s reflections on the reported 
evaluations, following are actions policymakers, 
practitioners, researchers, parents and community 
members can take to improve minority academic 
achievement. 

1 . Focus on Improved Academic Achievement and 
Outcomes for All. 

♦ National leaders should continue to build 
consensus around acceptable achievement 
gains and require that these gains be shown for 
all student groups. National attention should 
focus on achievement differences among the 
states and ways to eliminate these differences. 

♦ States should create benchmarks for 
improving academic achievement for all 
student groups and provide resources for 
school districts to attain those benchmarks. 
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States and school districts should support and 
maintain high quality leadership and ensure 
the adequate implementation of programs to 
enhance minority academic achievement. 

School districts and schools should expect 
high achievement from all students and 
provide academically demanding curricula that 
are meaningful and available across schools and 
grade levels to bring all students to higher levels 
of knowledge and achievement. 

States and localities should develop a multi- 
layered “check” of achievement using a 
variety of test measures, such as NAEP, state- 
mandated tests, Stanford-9 or ITBS; and also 
use indicators that provide a broad perspective 
about students, such as classroom-based 
assessments, attendance, behavior (disciplinary 
incidents), course enrollment and passing rates, 
types of courses completed and graduation 
rates, among other measures. 

School districts and schools should provide 
professional development and support to 
ensure that teachers (and other involved adults, 
as appropriate) have a deep understanding of 
curriculum, are familiar with innovative 
instructional methods, and have knowledge and 
interpersonal competence with cultures other 
than their own. 

Schools should provide students, families and 
communities with specific information on 
what constitutes high academic standards and 
support their expectations for excellence in the 
educational system. 

Families, youth advocates and communities 
should hold schools accountable for high 
levels of achievement for all students, reinforce 
academic skills learned both at home and at 
school, and ensure that every child has an 
advocate outside of the school system or 
program. 



2. States and Localities Should Provide the 
Necessary Supports to Ensure Student Success, 
including: 

♦ Reduced student-to-teacher ratios . A range of 
strategies should be employed by schools and 
programs to provide more personal teaching and 
learning environments to foster higher levels of 
academic achievement. These strategies may 
include smaller classes, small learning 
communities, teacher’s aides, team teaching, 
tutoring, mentoring and ancillary supports. 

♦ Extended learning time . To accelerate and 
reinforce student learning, programs should 
encourage or require additional time and 
opportunities (such as longer days, weekends 
and summer courses). 

♦ Long-term supports . Programs should 
encourage student participation over an 
extended time (two years or more) to create and 
sustain stable relationships between participants 
and knowledgeable adults, and to help youth 
make successful transitions as they progress up 
the educational ladder. 

♦ Scholarships and/or financial support . - 
Programs should provide financial support to 
youth as needed to motivate participation and 
persistence in quality educational experiences. 
Programs should also provide continual 
guidance to youth and monitor the impact of the 
funds on student achievement, retention and 
graduation. 

3. Start Early, Don’t Stop. 

♦ National leaders, states and school districts 
should prevent minority students from falling 
behind by expanding early childhood programs 
and providing continuous guidance and supports 
through the elementary and high school years. 

♦ National leaders, states and school districts 
should boost efforts to increase minority 
students 9 entry into and graduation from 
postsecondary education . 
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At almost every educational level, schools and 
community-based programs across the country are 
reporting good news about the academic 
achievement of the minority students they are 
serving. Although gaps overall are still large, and 
most reported achievement gains are small, these 
programs have proven there is every possibility of 
succeeding in raising achievement for all. 
Implementing the recommendations above could 



help the nation move beyond a feeling of 
helplessness regarding achievement gaps by 
providing specific information on program design 
and strategies about “what works” to enhance 
academic achievement. The larger challenge is 
creating the national will to set in place 
mechanisms that will eliminate differences in 
academic achievement among students correlated 
with race or ethnicity. 
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Educational researchers would prefer to think that 
their trade is a precise, scientific discipline with well- 
defined concepts and standardized procedures 
leading to uncontested results. However, between 
the ideal and reality there is usually a wide gap. 
Social phenomena are generally too complex to be 
isolated and measured, rigorous research methods 
may clash with ethical concerns, and the search for 
objectivity may be clouded by program advocacy. 
Good researchers strive for a balance between what 
should be done (the “perfect” research) and what 
can be done. For those dealing with secondary 
sources, that is, research done by others, the 
negotiation between ideal and reality is even more 
frustrating. Jargon-laden research must be decoded 
into intelligible language, large amounts of work 
must be reviewed to select a few evaluations for 
inclusion, and at the end, the questions that 
propelled the search may remain unanswered. 

The making of this report reflects all these 
challenges. The journey that started 18 months ago 
required reviews of hundreds of articles, reports, 
books, unpublished manuscripts, and other 
documents to produce the summaries included in 
this report. This chapter briefly describes the path 
traveled, its obstacles and discoveries along the way. 
(For a description of the report methodology, see 
Overview and Research Note) 

The Journey and Its Obstacles: 

The U.S. is perpetually awash in ‘ new 9 and 
self-proclaimed ‘ highly effective ’ programs for 
improving students' academic achievement . . . 
The evidence that most of these programs 
‘ work 9 has always been modest , and evidence 
of generalizability of effects is, for the 
majority of programs, non-existent ( Sam 
Stringfield 1 ). 



Finding evaluations of any quality is a difficult task, 
except for federal initiatives or grantee programs 
that mandate such studies. Program evaluation is a 
time-consuming process that may take money away 
from direct services. For many educators and 
youth program practitioners, already struggling with 
funding shortages, the idea that some of this money 
will be diverted from services to support research is 
anathema. However, without research, program 
practitioners may be perpetuating failing or 
mediocre interventions whose long-term 
consequences are much costlier to the young people 
and society. Although common sense indicates that 
interventions without a proven record of success 
should not be replicated, the search for the “magic 
solution” seems to overcome common sense. A 
non-scientific estimate of the literature search 
suggests a ratio of five “how to” reports - that is, 
reports on how to implement a specific but often 
untested intervention - to one evaluation of a 
program or strategy. 

The search process for this report was particularly 
challenging, more so than for the two previous 
AYPF compendia. Over 200 documents were 
collected for an initial selection of less than 50. As 
described in the Research Note , the acceptance of 
evaluations for the report was dependent on five 
criteria that included population, measurements, 
methodology, length of research and scope. The 
following paragraphs discuss some of the obstacles 
encountered in satisfying the criteria and how they 
influenced this report’s outcome. For readers who 
are interested in research but not familiar with its 
basic terminology and standards, a brief explanation 
is provided at the end of the chapter, under the title 
“ Basic Principles of Educational Research .” 
Definitions of research terms are included in the 
Glossary. 
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Population 

The most important caveat about the documents 
reviewed was their treatment of the population. 

First, although the initial purpose was to include 
evaluations and programs for youth from all 
minority racial/ethnic groups, the final report 
includes few studies related to Native American and 
Asian/Pacific Island youth. The report’s primary 
emphasis on African American and Latino youth 
reflects rather a lack of information on the other 
groups than a search process that focused on these 
two groups. 

Second, most evaluations report on the student 
population as a homogeneous group, where 
demographics, such as race/ethnicity, appear as part 
of a description, but are rarely taken into account in 
the analysis. Few evaluations disaggregated their 
findings by sub-groups - ethnicity, gender, socio- 
economic status, English proficiency or baseline 
academic achievement. Disaggregating data 
requires more work during the data collection phase, 
demands a larger pool of students to provide 
statistically meaningful results, and risks exposing 
program weaknesses. However, this type of 
analysis is essential to highlight areas that require 
improvement and areas of proven success, thus 
offering key information for school administrators 
and program implementers. 

The evaluations of Chapel Hill-Carrboro City 
Schools (CHCCS) and GE Fund College Bound are 
good examples of the value of disaggregating data. 
Results from the CHCCS program showed 
improved levels of proficiency in mathematics and 
reading for African American students with a 
reduction in the score gap between these students 
and their white peers. Yet the writing scores for 
African American students actually declined during 
the period of the study. The evaluation for the GE 
Fund indicated an overall increase in college 
enrollment rates for all participants, but more so for 
Latinos. However, the gap in enrollment between 
African Americans and whites increased. Faculty 
involved with the two projects can use the data to 
examine their strategies toward each group of 
students, to replicate the strategies that are boosting 
minority achievement and revise the strategies that 



are not working. Programs that claim success 
without disaggregating their data may be helping one 
group of students while the other groups continue to 
fail. In fairness to the student population as a 
whole, these programs are not achieving their 
objectives. 

Outcome Measurements 
The initial criteria for acceptance of evaluations 
required a set of outcome measures that would 
provide a broad picture of the students’ 
performance, such as test scores, number and type 
of credits taken, GPA, dropout and attendance 
rates, as well as postsecondary education or 
employment data. This requirement was based on 
the principle that relying on a single measure to 
assess a program may lead to incomplete, and many 
times, misguided conclusions. For instance, the 
evaluation of Equity 2000 , a program that proposes 
academically challenging curricula for all high school 
students, shows a 30% increase in student 
enrollment in advanced mathematics classes. It also 
shows an increase of about 50% in failure rates in 
these same classes. While the enrollment data 
suggest an accomplishment, the data on passing 
rates indicate the need for much work before the 
program claims success. 

Despite efforts during the search period, few 
evaluations reported more than two measures of 
achievement, the most frequent being test scores. It 
is inadvisable to use tests as the sole measure of 
student knowledge for many reasons. For instance, 
multiple-choice tests measure only one type of 
learning (memorization); some tests have been 
criticized as being culturally biased against minority 
students; some students are great test-takers while 
others are not; tests evaluate the student on one day 
out of 180 or more per school year, and on one set 
of specific competencies; tests do not necessarily 
assess the students’ mastery of essential skills, such 
as problem solving, communicating complex ideas, 
using different strategies to reach a solution, or 
working in groups. 2 Notwithstanding the myriad 
problems with testing, the reality is that tests are 
being used across the country as a measure of school 
accountability and student achievement, and as 
gateways to advancement along the educational ladder. 
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Programs that raise the test scores of minority youth 
do increase the youths’ chances of high school 
graduation, college admission and success in later life. 

Acceptance of test scores as a valid measure of 
student achievement does not solve the question of 
whether it should be the sole measure. There are 
many methodological limitations associated with an 
overemphasis on test scores, such as: 

♦ Habituation - Although questions change with 
different administrations of a test, students get 
used to the logic behind the test and its style. 
With time, scores in that specific test tend to go 
up due to habit, rather than actual improvement 
of student performance. 

♦ Lack of reliability - Few of the current 
statewide tests are submitted to statistical 
analyses to assess their validity and reliability. 3 

♦ Political pressure - Tests may be weakened to 
address parental opposition and, in this case, 
increased test scores within a period of time 
may reflect a change in the tests (becoming 
easier or lowering the cut-off scores) rather than 
better-prepared students. 

♦ Teaching to the test - Higher test scores may 
reflect the schools’ emphasis on teaching to the 
test. With teachers focused on preparing the 
students to take the test, it is expected that 
scores will go up, even if the students still miss 
important competencies for future careers, 
sacrifice depth for breadth, and do not work on 
problem solving and critical thinking skills 
important for democratic citizenship and the 
new job market. 

♦ <( Cheating ” the system - Higher test scores 
may also hide an increase in dropout rates or in 
the number of students identified as having 
limited English proficiency or in need of special 
education (generally, these students are 
exempted from statewide tests). As the students 
who test poorly for various reasons are pushed 
out of the system, the average scores of the 
remaining students increase. Without other 



measurements, such as trend data on special 
education enrollment, dropout rate, college 
attendance and retention, enrollment in remedial 
courses in college, or type of employment, 
conclusions based solely on test scores are 
limited. 

All this being said, with the current emphasis on 
testing it is understandable that researchers rely on 
tests to evaluate the success of a program. Indeed, 
the vast majority of evaluations found used test 
scores as the sole measure (at least, the sole 
quantifiable measure) to assess a program’s 
performance. 

Evaluations that use scores on only one test to 
assess a program create a serious obstacle for 
comparisons across programs. For example, is a 
30% increase on the Texas Assessment of 
Academic Skills (TAAS) a greater feat than a 10% 
increase on the California Achievement Test (CAT)? 
Evaluations using the National Assessment of 
Educational Progress (NAEP) can be compared, 
since this is a nationwide assessment (although the 
NAEP is not conducted yearly and scores are not 
reported for individual students), but few studies 
reviewed used NAEP data. 4 Another question that 
remains unanswered by a raw test score is its 
impact on the student’s life. What does a three- 
point increase in a test represent for the student? Is 
this student now at the expected grade level? How 
much more does the student need to be proficient in 
the subject? 

Translating results into grade levels or percentiles 
facilitates comparisons. For instance, after the 
Calvert model was implemented at the Dr. Carter 
Goodwin Woodson Elementary School, an all 
African American inner city school in Baltimore, the 
first grade average reading comprehension scores 
went up 31 points, from the 18 th to the 49 th 
percentile. This measure indicates that before 
Calvert was implemented, Woodson students were 
scoring on average below 82% of all students who 
took the Maryland test. One year into the new 
program, the average score of Woodson students 
placed them close to the middle. This information 
does not answer the question of how well the test 
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assessed what students need to know to succeed in 
life, but very few tests, if any, have such predictive 
power. 5 

Methodology 

Design - Methodological rigor should be a concern 
for any researcher, but the standards of rigorous 
research are not so clear in the educational field. 
Evaluations using control or comparison groups 
were rarely found in the search for this report. The 
majority of the documents that we found compared 
the program or school with existing databases at the 
district or state levels. As the methodological rigor 
weakens, the findings become less reliable or 
generalizable, and the research process turns into an 
expensive, but fruitless exercise. Researchers who 
Ideal with limited budgets must carefully choose a 
design that provides the required information 
without unjustifiable expense. Interestingly enough, 
despite complaints of lack of research funding, the 
search produced a number of evaluations with 
highly complex, costly but inefficient designs. 

Use of indicators - To evaluate performance 
changes, the data collected must be compared 
against either a baseline performance (how the 
students performed before the program) or some 
established indicator (how the students were 
expected to perform). A claim that 70% of Latino 
students in a program graduated is meaningless 
without information on how many students 
graduated before the program, or the overall 
graduation rate for Latino students in that specific 
school district or state. An enrollment of 80% in an 
algebra class may seem high until we discover that 
algebra is a mandatory course for graduation in that 
school district, and the enrollment should be 100%. 
Numbers gain meaning only within a context. This 
comment should be obvious, but a number of 
rejected evaluations claimed the success of a 
program without that context. 

Statistical treatment of data - In addition to 
including baseline data and/or contextual indicators, 
researchers should calculate the statistical 
significance of their findings. A 12% decline in the 
test score gap between African Americans and white 
students in a specific program could reflect either 



the positive impact of the program or normal 
fluctuations in test scores. Statistical tests are 
needed to separate random occurrences from 
treatment effects. If these test scores are 
performed, researchers must report results, 
including levels of significance. Again, reporting 
statistical significance is a basic research principle 
that was frequently forgotten among the documents 
reviewed. 

Researcher bias - It is not uncommon in the 
educational field that a research institution or an 
individual researcher monopolizes the evaluation of 
specific programs or initiatives. In an ideal world, 
third parties (“outsiders”) with no direct interest in 
the program should conduct the evaluation to ensure 
the impartiality of analysis. In reality, however, it is 
often cheaper and easier for an “insider” or 
advocate with the appropriate research skills to 
conduct an internal evaluation. Fortunately, the 
review conducted for this report shows that 
“insider” evaluations can be just as rigorous and 
impartial as third-party evaluations. For example, 
many school evaluations are conducted through 
school district staff. Depending on the local political 
climate, these studies can be quite independent, 
particularly when they are intended as internal tools 
of assessment. The Chapel Hill-Carrboro City 
Schools evaluation is an example of an impartial 
insider research. In contrast, a number of 
“outsider” evaluations were rejected because they 
contained blatantly biased analysis. 

Scope 

If we do not describe the possible dystopias we 
shall be left only with [our] utopias. If we do 
not insist on bringing research findings (which 
may be politically inconvenient ) into the 
public arena , we contribute to the erosion of 
democracy ( Gipps 6 ). 

It is well-known that academic journals in any 
science (not only education) tend to publish 
evaluations that show success, while studies with 
negative findings are politely rejected. To ensure a 
more balanced perspective of programs geared 
toward minorities, the search included manuscripts 
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and unpublished grant reports in addition to 
published articles. Yet, whether the evaluation was 
published or not made little difference. A tendency 
to spin results into success or hide less than 
successful results was common to the majority of 
the documents. Chapel Hill- Carrboro City 
Schools , GE Fund College Bound and High 
Schools That Work deserve commendation for the 
courage to show accomplishments and 
shortcomings. Without this courage, program 
evaluation becomes little more than statistical 
cheerleading. Evaluators who hide negative results 
or use their trade as a tool for ideological positions 
are doing a disservice to policymakers, who will 
make decisions based on questionable information. 
By perpetuating misinformation, these evaluators 
are also doing a disservice to the educational 
process and to the youth, victims of failed strategies 
disguised as success. 

( : \ 

The first conclusion resulting from the search 

process is that the most useful research is 
based on simple but methodologically sound 
design and provides information that is clear 
and easy to understand. This type of 
information is essential for educators and 
program practitioners who need to convince 
skeptics, placate critics, or expand support for 
their programs. Less useful are 
methodologically unsound evaluations, or 
evaluations that are so complex and hard to 
read that, high quality or not, they provide little 
usable information to policymakers and 
practitioners. 

V J 

Report Overview 

A brief overview of the evaluations selected for this 
report reflects the following characteristics: 

♦ Range. The selected evaluations present a 
mix of policy initiatives and public or private 
programs. Together, the summaries span the 
educational ladder, from early childhood to 
graduate education. Although some district- 
wide reforms address all grades from K-12, 
evaluations of programs or initiatives that 
specifically target middle school students 
were not found. The search, albeit quite 
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extensive, may have missed such programs, 
but this finding is worrisome, since many 
students who drop out of school start falling 
behind in middle school. 

Population. Few programs and initiatives 
target specific racial/ethnic groups. The 
majority serve a large number of minority 
students for two basic reasons. First, the 
majority of evaluations dealt with programs 
and initiatives targeting Title I schools, that is, 
schools with large numbers of students living 
at or below the poverty level. Although 
poverty is by no means an exclusive problem 
of minorities, minority children and youth are 
over-represented among the poor. Second, 
some programs are located in areas where a 
specific minority group predominates, such as 
Latinos in Puerto Rico and some schools 
districts in California, and African Americans 
in Washington, DC, and Baltimore. The 
Population textbox in each of the evaluation 
summaries in Section II reports the 
population in each study by racial/ethnic 
group, income level, geographical location, 
and program targeted level. 

Methodology. The studies summarized in this 
compendium vary in design and 
methodological rigor. Nineteen out of the 38 
summaries use a control or comparison 
group, four are longitudinal studies, nine 
employ the pre/post-treatment method and 
eleven compare their findings against district, 
state or national databases (some use more 
than one method). Four summaries are 
descriptive only. 

Measures. For K-12 programs, test scores are 
the most common measure of academic 
achievement. Most evaluations rely on one 
type of test, often the state-mandated test. A 
few studies use standardized tests adopted 
nationwide, such as the Iowa Test of Basic Skills 
(ITBS) and the Stanford-9 (SAT-9). 7 Among 
other indicators, high school programs frequently 
cite college enrollment data, while postsecondary 
education programs look at retention rates. Few 
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reports provide data on employment, including 
Tribal Colleges , Compact for Diversity and the 
three long-term studies of early childhood 
programs ( Abecedarian , Child Parent Centers 
and High/Scope ). 8 

This analysis discussed utopias and dystopias, the 
politically inconvenient but statistically significant. 
The hope is to contribute information that can guide 
educators and policymakers to better informed 
choices of strategies and initiatives that improve the 
academic achievement of minority youth; and foster 
a better understanding of the need for evaluation 
studies that look at facts, rather than dreams, and 
reality, rather than rhetoric. This hope is translated 
in the recommendations below. 

Recommendations 

♦ A large-scale , national and comprehensive 
educational research agenda must be 
developed to (a) determine which strategies and 
policies have resulted in the most benefit, for 
whom, and at what cost, (b) provide guidance 
to evaluators on what type of research would be 
most useful to policymakers and practitioners, 



and (c) provide guidance to practitioners on 
why quality research is needed, how to initiate it 
and use it. 

♦ Public and private funding sources must 
require and support high quality program 
evaluations and utilize findings to improve 
policy and practice, rather than to punish 
programs. 

♦ Data must be disaggregated by race , ethnicity, 
limited English proficiency, disability status, 
gender and poverty level and be made publicly 
accessible to researchers, educators, 
policymakers, families and the public at large. 

♦ Researchers should look into a range of 
achievement indicators including, numbers of 
students enrolled and dropping out, attendance, 
test scores, GPAs, graduation, suspensions, 
expulsions, and special education referrals. 

They should also translate their findings into 
language that is accessible to policymakers, 
practitioners, educators, families and students, 
so that research findings can be translated into 
better education policies and practices. 



Addendum: Basic Principles of 
Educational Research 

The next paragraphs attempt to provide readers 
who are not familiar with research with some very 
basic tools to help them navigate the summaries 
and use the findings to make their own assessment 
about the programs. These paragraphs reflect the 
many discussions about research among the 
members of the editorial team. However, its 
inclusion is not without a certain hesitation since 
a large amount of information is necessarily omitted. 

Control Groups 

The use of control groups provides the most 
rigorous design to assess the effect of an 
intervention, but it also raises important ethical 
questions. In educational research that uses control 
groups, two groups of individuals are randomly 
selected - one group attends the program (treatment 



group) and the other does not (control group). 

When using a control group, the researcher ensures 
that the two groups are as similar as possible and 
limits the factors that may interfere with the 
education process. This control enables the 
researcher to attribute later differences between the 
treatment and the control groups to the program’s 
effect with some degree of certainty (total certainty 
is an unattainable ideal). However, a control group 
supposes that the evaluators, with the consent of 
program directors or implementers, made a choice 
to provide a strategy that may help a group of needy 
youth while refusing it to another needy group, a 
difficult decision for any concerned individual. 
Programs that have more applicants than openings and 
select students through a lottery process have a natural 
control group in the students who do not win the 
lottery. The lottery is a totally random process that 
excludes the possibility of personal bias from admission 
personnel, but few programs use this system. 
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Comparison Groups 

Evaluators can solve this problem in part by finding 
a comparison group , that is, an existing group of 
students similar to the treatment group who will not 
attend the program. For instance, students in two 
schools that are demographically and academically 
similar where one school implements the program 
and the other does not. A popular comparison in 
educational research is between students in a 
specific program and district wide, statewide or 
nationwide data. This type of comparison group is 
the easiest to identify, because the data already 
exists, but is the least reliable, since large databases 
include schools with different academic 
achievement, socio-economic background, type of 
personnel, and funding levels. 

Matching 

Control and comparison groups must be matched 
for demographics, socio-economic status, and prior 
academic performance to ensure that they are 
similar. If the groups are not matched on all these 
factors, the evaluators cannot infer whether the 
findings reflect program effect or the initial 
differences between the groups. A treatment group 
starting at a higher academic level than the 
comparison or control group is more likely to show 
higher scores even without the program. Or the 
converse may be true. The treatment group may have 
more students who are struggling academically. In this 
case, results may favor the control or comparison 
group even if the program is working. Although this 
explanation appears obvious, we found evaluation 
studies that claimed program success based on 
comparisons of groups that differ in their basic 
demographics and performance characteristics. 

Pre- and Post-Treatment Data 

Research using pre- and post-treatment data does 
not have the problem of group differences, but 
brings up other concerns, such as differences in 
tests used to measure progress, natural student 
maturation, or interferences due to the exit and 
entrance of students, changes in school personnel, 
and other factors. 
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Timing and Longitudinal Studies 
Time is an important factor in evaluations. A study 
conducted too early, before the strategies are fully 
implemented, will not show clear results. Studies 
where the data is collected only once do not provide 
information about the program’s ability to promote 
changes on an ongoing basis. It is not unusual that 
a program shows positive short-term changes as a 
result of the attention generated during its initial 
implementation. If this is the case, results may 
decline the following year, when the novelty has 
passed and the attention wanes. Longitudinal 
studies provide the best information to assess the 
program’s performance. However, longitudinal 
studies are both difficult to implement and 
expensive. In addition, as the time passes, contact 
with research participants becomes more difficult, 
the initial treatment and control group dwindle, and 
results from such small samples become less prone 
to generalization. Th t Abecedarian Project, Child 
Parent Centers and High/Scope Perry Preschool 
are examples of the advantages and difficulties of 
long-term longitudinal research. 

Use of Samples 

In research, population is the generic name for what 
is being studied (it can be rats, as in experimental 
psychology research, as well as schools, students 
and teachers). Studies of small programs that exist 
in one school should include all the students as the 
results will be more reliable. However, for large 
studies, such as programs implemented in many 
schools or large school districts, it may become 
impossible to manage the study using the whole 
student population and the use of samples becomes 
imperative. In general, samples are randomly 
selected using some type of lottery, computer- 
generated numbers, or similar process. Researchers 
can also select samples to answer specific research 
questions. For instance, they can select only the 
best schools in a district to compare with the best 
schools in another district, or they can select only 
male students to analyze how a program affects 
males. When researchers select the sample, they 
should explain their selection process. 
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